JP2015049253A - Voice synthesizing management device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To reduce a work burden of a user who sets a variable applied to voice synthesizing.SOLUTION: A display control part makes a display device display an adjustment image 60 which contains an adjustment region 62 in which a first axis A1 representing mixing ratio between a first voice and a second voice at the time of synthesizing voices to be synthesized and a second axis A2 representing sound volume of the voice to be synthesized are set. An instruction accepting part accepts, from a user, instructions about an instruction point X1 and an instruction point X2 in the adjustment region 62. An information management part generates control information which represents temporal change in mixing ratio from a numerical value corresponding to the instruction point X1 on the first axis A1 to a numerical value corresponding to the instruction point X2 and a temporal change in sound volume from a numerical value corresponding to the instruction point X1 on the second axis A2 to a numerical value corresponding to the instruction point X2.

Description

  The present invention relates to a technique for managing variables applied to speech synthesis.

  Conventionally, a unit connection type speech synthesis technique for synthesizing desired speech using a set of a plurality of speech units (hereinafter referred to as “speech library”) collected from pre-recorded speech has been proposed. . For example, Patent Document 1 synthesizes speech having a voice quality different from that of an existing speech library by mixing (morphing) each speech unit of two types of speech libraries corresponding to speech of different voice qualities. Techniques to do this are disclosed.

Japanese Patent Laid-Open No. 9-50295

  By the way, the volume of each speech unit may be different for each speech library. Therefore, the volume of the synthesized speech generated by mixing a plurality of speech units varies depending on the mixing ratio of each speech unit. For example, while decreasing the proportion of speech units in an audio library that tends to be louder over time, the proportion of speech units in an audio library that tends to be lower in volume is increased over time to mix the two. In this case, the volume of the synthesized speech decreases with time. Therefore, in order to keep the volume of the synthesized voice constant, it is necessary for the user to adjust the time change of the volume of the synthesized voice so as to be interlocked with the time change of the mixing ratio, and the work burden on the user is heavy. There's a problem.

  In the above description, the sound volume is focused for the sake of convenience, but the same situation can be applied to the acoustic characteristics other than the sound volume. For example, the ratio of speech units (sounds with a clear and bright atmosphere) in an audio library whose pitch tends to be perceived easily on the high frequency side is reduced over time, and the pitch tends to be easily perceived on the low frequency side. If the ratio of speech segments in the speech library (for example, unclear and dark atmosphere speech) is increased over time and mixed, the pitch perceived by the listener decreases over time. Therefore, in order to keep the pitch of the synthesized speech constant, the user needs to adjust the time variation of the pitch of the synthesized speech so as to be interlocked with the time variation of the mixing ratio. In view of the above circumstances, an object of the present invention is to reduce a work burden on a user who sets a variable applied to speech synthesis.

  In order to solve the above problems, the speech synthesis management device according to the first aspect of the present invention includes a first axis indicating a mixing ratio of the first speech and the second speech at the time of synthesis of the synthesis target speech, and a synthesis target. Display control means for displaying on the display device an adjustment image including an adjustment area in which a second axis indicating the first characteristic variable relating to the acoustic characteristics of the sound is set; and the first indication point and the second indication point in the adjustment area An instruction receiving means for receiving an instruction from the user, a time change in the mixing ratio from a numerical value corresponding to the first indicating point on the first axis to a numerical value corresponding to the second indicating point, and the first instruction on the second axis Information management means for generating control information indicating a time change of the first characteristic variable from a numerical value corresponding to the point to a numerical value corresponding to the second indication point. In the above configuration, the first indication point and the first indication from the user within the adjustment region in which the first axis indicating the mixing ratio of the first voice and the second voice and the second axis indicating the first characteristic variable are set. 2 indicating points are indicated, and the time change of the mixing ratio from the numerical value corresponding to the first indicating point on the first axis to the numerical value corresponding to the second indicating point and corresponding to the first indicating point on the second axis Control information indicating the time change of the first characteristic variable from the numerical value to the numerical value corresponding to the second indication point is generated. Therefore, the user can instruct the time change of the mixing ratio and the time change of the first characteristic variable in parallel. That is, there is an advantage that the work burden on the user can be reduced as compared with the conventional configuration in which it is necessary to individually indicate the time change of the mixing ratio and the time change of the first characteristic variable.

  In a preferred aspect of the present invention, the information management means calculates the second period from the numerical value corresponding to the first indication point on the first axis for the period from the first time point to the second time point on the time axis of the synthesis target speech. Control showing the time change of the mixing ratio to the numerical value corresponding to the indicated point and the time change of the first characteristic variable from the numerical value corresponding to the first indicated point to the numerical value corresponding to the second indicated point on the second axis. Generate information. In the above aspect, the mixing ratio and the temporal change of the first characteristic variable can be adjusted for a specific period from the first time point to the second time point in the synthesis target speech. Further, according to the configuration in which the instruction accepting unit accepts the instructions at the first time point and the second time point from the user, the time change between the mixing ratio and the first characteristic variable is adjusted for the user's desired period of the synthesis target voice. There is an advantage that you can.

  In a preferred aspect of the present invention, the display control means includes an adjustment image in which each point in the adjustment area is set to a different display aspect, and the first in the adjustment area from the first time point to the second time point on the time axis. A variable image including a transition image that changes from a display mode at the indicated point to a display mode at the second indicated point is displayed on the display device. According to the above configuration, since the transition image that changes from the display mode at the first indication point in the adjustment region to the display mode at the second indication point from the first time point to the second time point is displayed on the display device, There is an advantage that the user can visually and intuitively grasp the mixing ratio and the temporal change of the first characteristic variable from the first time point to the second time point. In addition, according to the configuration in which the display control unit displays on the display device the variable image representing the time change of the second characteristic variable related to the acoustic characteristic of the synthesis target speech in the shape of the transition image, the time change of the second characteristic variable is changed. The display content can be simplified as compared with a configuration in which the image is displayed separately from the image.

  In a preferred aspect of the present invention, the display control means includes an adjustment image in which each point in the adjustment area is set to a different display aspect, and a musical note iconic image representing each note of the synthesis target voice, a time axis and a pitch. The score image arranged in the score area in which the axis is set is displayed on the display device, and the display mode of each point on the time axis in each note image is changed from the first designated point to the second designated point in the adjustment area. The display mode at the point corresponding to the point in the route is set. In the above aspect, the musical note image representing each note is also used for displaying the mixing ratio and the time variation of the first characteristic variable. Therefore, the relationship between the time variation of the mixing ratio and the first characteristic variable and each note is expressed as follows. There is an advantage that the user can easily grasp. In addition, the specific example of the above aspect is later mentioned as 3rd Embodiment, for example.

  In a preferred aspect of the present invention, the display control means corresponds to an adjustment image in which each point in the adjustment region is set to a different display form, a note graphic image representing each note of the synthesis target voice, and each musical note graphic image. A score image arranged in a score area in which a time axis and a pitch axis are set is displayed on the display device, and the display mode of each point on the time axis in each auxiliary image is displayed in the adjustment area. It sets to the display mode in the point corresponding to the said point among the paths from the 1st indication point to the 2nd indication point. In the above aspect, since the auxiliary image corresponding to each note image is used to display the mixing ratio and the time change of the first characteristic variable, the relationship between the time change of the mixing ratio and the first characteristic variable and each note is used. There is an advantage that the person can easily grasp. In addition, the specific example of the above aspect is later mentioned as 4th Embodiment, for example.

  The speech synthesis management device according to the second aspect of the present invention includes an adjustment region in which a reference point corresponding to each of N types (N is a natural number of 3 or more) of speech used for synthesis of speech to be synthesized is set. Display control means for displaying the adjustment image on the display device, instruction receiving means for receiving instructions of the first and second instruction points in the adjustment area from the user, and N kinds of sounds at the time of synthesizing the synthesis target sound An information management means for generating control information indicating a time change from a numerical value corresponding to the first indicating point to a numerical value corresponding to the second indicating point in the adjustment region with respect to the mixing ratio. With the above configuration, the first indication point and the first indication from the user within the adjustment area in which the reference points corresponding to each of N types (N is a natural number of 3 or more) of speech used for synthesis of the synthesis target speech are set. Two indication points are instructed, and control information indicating a time change from a numerical value corresponding to the first indication point to a numerical value corresponding to the second indication point in the adjustment region is generated for the mixing ratio of N types of sounds. Accordingly, the user can instruct a change in the mixing ratio over time while visually confirming the mutual relationship between the sounds in the adjustment area. That is, there is an advantage that the user's work load can be reduced as compared with the configuration in which the mixing ratio of N types of voices needs to be individually indicated.

  The speech synthesis management device according to each aspect described above is realized by hardware (electronic circuit) such as DSP (Digital Signal Processor) dedicated to generation of control information, etc., and general purpose such as CPU (Central Processing Unit) This is also realized by cooperation between the arithmetic processing unit and the program. The program of the present invention can be provided in a form stored in a computer-readable recording medium and installed in the computer. The recording medium is, for example, a non-transitory recording medium, and an optical recording medium (optical disk) such as a CD-ROM is a good example, but a known arbitrary one such as a semiconductor recording medium or a magnetic recording medium This type of recording medium can be included. For example, the program of the present invention can be provided in the form of distribution via a communication network and installed in a computer. The present invention is also specified as an operation method (speech synthesis management method) of the speech synthesis management device according to each aspect described above.

1 is a block diagram of a speech synthesizer according to a first embodiment of the present invention. It is explanatory drawing of a mixing process. It is a schematic diagram of synthetic information. It is a flowchart of a speech synthesis process. It is a schematic diagram of an edit image. It is a flowchart of operation | movement of a speech synthesizer. It is a flowchart of an edit process. It is a schematic diagram of an edit image. It is a schematic diagram which shows the relationship between an adjustment area | region and each indication point. It is a schematic diagram which shows the relationship between an adjustment area | region and each indication point. It is a schematic diagram of an edit image. It is a schematic diagram of the edited image in 2nd Embodiment. It is a schematic diagram of the edited image in 3rd Embodiment. It is a schematic diagram of the edited image in 4th Embodiment. It is explanatory drawing of the mixing process in 5th Embodiment. It is a schematic diagram of the edited image in 5th Embodiment. It is explanatory drawing of the mixing ratio in 5th Embodiment. It is a schematic diagram of the adjustment image in 6th Embodiment. It is a schematic diagram of the modification of the path | route between each indication point. It is a schematic diagram of the modification of the path | route between each indication point. It is a schematic diagram of the modification of an adjustment area | region. It is a schematic diagram of the adjustment area | region in the modification of 5th Embodiment. It is a schematic diagram of the adjustment area | region in the modification of 5th Embodiment.

<First Embodiment>
FIG. 1 is a block diagram of a speech synthesizer 100 according to the first embodiment of the present invention. The speech synthesizer 100 generates a speech signal S of an arbitrary speech by a unit connection type speech synthesis process that connects a plurality of speech units. Specifically, the speech synthesizer 100 of the first embodiment is a signal processing device that generates a speech signal S of a singing voice of an arbitrary music (hereinafter referred to as “synthetic music”), and includes an arithmetic processing device 10 and a storage device. 12, a display device 14, an input device 16, and a sound emitting device 18 (for example, an information processing device such as a mobile phone or a personal computer).

  The display device 14 (for example, a liquid crystal display panel) displays an image instructed from the arithmetic processing device 10. The input device 16 is an operation device (for example, a pointing device such as a mouse or a keyboard) operated by the user for various instructions to the speech synthesizer 100, and includes a plurality of operators operated by the user, for example. Is done. Note that a touch panel configured integrally with the display device 14 may be employed as the input device 16. The sound emitting device 18 (for example, a speaker or headphones) reproduces sound according to the audio signal S. Illustration of a D / A converter for converting the audio signal S from digital to analog is omitted for convenience.

  The storage device 12 stores a program executed by the arithmetic processing device 10 and various data used by the arithmetic processing device 10. A known recording medium such as a semiconductor recording medium or a magnetic recording medium or a combination of a plurality of types of recording media is arbitrarily employed as the storage device 12. The storage device 12 of the first embodiment stores an audio library L (LA, LB) and synthesis information Q.

  The speech library L is a set of a plurality of speech segments P (PA, PB) collected in advance from the speech of a specific speaker. Each speech element P is a phoneme (for example, a vowel or a consonant) that is a minimum unit of linguistic meaning distinction, or a phoneme chain (for example, a diphone or a triphone) that connects a plurality of phonemes. Each speech segment P is expressed as a sample sequence of a speech waveform in the time domain or a time sequence of a spectrum in the frequency domain calculated for each frame of the speech waveform.

  As illustrated in FIG. 1, the storage device 12 of the first embodiment stores a plurality of audio libraries L (LA, LB). The speech library LA is a set of a plurality of speech segments PA extracted from the first speech, and the speech library LB is a set of a plurality of speech segments PB extracted from the second speech. The first voice and the second voice have different voice qualities (voice colors). Specifically, the first voice (each voice segment PA) and the second voice (each voice segment PB) are voices uttered by different speakers, or one voicer has a different voice quality. It is the voice that was uttered.

In the first embodiment, as illustrated in FIG. 2, the speech segment P (PA, PB) corresponding to the pronunciation content of the speech to be synthesized (hereinafter referred to as “synthesized speech”) is represented by the speech library LA and the speech library. A speech unit PA selected from both of the LBs and sequentially selected from the speech library LA and a speech unit PB selected from the speech library LB are mixed at a mixing ratio R (hereinafter referred to as “mixing process”). A speech segment PS is generated. In the mixing process (morphing), for example, the variable pA related to the voice quality of the speech unit PA and the variable pB related to the voice quality of the speech unit PB are weighted and added according to the mixing ratio R as expressed by the following formula (a). This is a process for calculating the variable pS relating to the voice quality of the speech element PS. The variable relating to the voice quality can be exemplified by a feature amount that defines the envelope of the voice spectrum.
pS = (1-R) · pA + R · pB (a)

  The mixing ratio R is the degree of dominance of each speech unit P in the mixing process of the speech unit PA in the speech library LA and the speech unit PB in the speech library LB (the degree reflected in the speech unit PS after mixing). ). Specifically, when the mixing ratio R is a minimum value (for example, 0), the mixed speech unit PS matches the speech unit PA, and the larger the mixing ratio R, the speech unit for the speech unit PS. When the influence of the segment PA is reduced and the mixing ratio R is the maximum value (for example, 1), the speech unit PS after mixing coincides with the speech unit PB. As understood from the above description, the voice quality of the speech segment PS can be set to an intermediate voice quality between the speech segment PA and the speech segment PB according to the mixing ratio R. A speech signal S is generated by interconnecting speech segments PS sequentially generated by the mixing process on the time axis. In addition to the mathematical expression (a), a known technique can be arbitrarily adopted for the mixing process of the speech element PA and the speech element PB.

  The synthesis information Q in FIG. 1 specifies a synthesis target voice. As illustrated in FIG. 3, the synthesis information Q of the first embodiment includes music information QM and control information QC. The music information QM is time-series data that specifies the content of the composite music, and specifies the pitch q1, the pronunciation period q2, and the voice code q3 for each note constituting the composite music. The pitch q1 is a note number based on, for example, the MIDI (Musical Instrument Digital Interface) standard. The pronunciation period q2 is a note duration defined by, for example, the start time and duration (or end time) of pronunciation. The voice code q3 designates the pronunciation content of the synthesis target voice (that is, the lyrics of the synthesized music). For example, a character (grapheme) constituting the lyrics of the synthesized music and a phoneme symbol of the phoneme corresponding to each character are designated as the speech code q3.

  The control information QC in FIG. 3 specifies a time change of a variable applied to the speech synthesis process. The control information QC of the first embodiment designates the time change of the mixing ratio R of the speech unit PA and the speech unit PB and the time change of the first characteristic variable. The first characteristic variable is a variable (feature amount) related to the acoustic characteristic of the synthesis target speech. In the first embodiment, the volume V is exemplified as the first characteristic variable. The control information QC of the first embodiment designates the temporal change of the mixing ratio R and volume V within a specific period (hereinafter referred to as “control period”) of the synthesis target sound (synthesized music).

  The arithmetic processing unit 10 (CPU) in FIG. 1 executes a program stored in the storage unit 12 to thereby execute a plurality of functions (instruction receiving unit 22, display) for editing the synthesis information Q and generating the audio signal S. The control unit 24, the information management unit 26, and the speech synthesis unit 28) are realized. A configuration in which each function of the arithmetic processing device 10 is distributed to a plurality of devices, or a configuration in which a dedicated electronic circuit (for example, DSP) realizes a part of the functions of the arithmetic processing device 10 may be employed.

  The voice synthesizer 28 generates a voice signal S by voice synthesis processing using the voice library L and the synthesis information Q stored in the storage device 12. FIG. 4 is a flowchart of the speech synthesis process. When the voice synthesizing process is started, the voice synthesizing unit 28 selects the voice element P (PA, PB) corresponding to the voice code q3 designated by the music information QM of the synthesis information Q for each note in the voice library LA and the voice library LB. Selections are made sequentially from both sides (SA1).

  The voice synthesizing unit 28 applies a mixing ratio R specified by the control information QC of the synthesis information Q for the current time for the voice element PA selected from the voice library LA and the voice element PB selected from the voice library LB. To generate a speech segment PS (SA2). Further, the speech synthesizer 28 adjusts the volume of the speech unit PS after mixing to the volume V specified by the control information QC for the current time (SA3). Then, the speech synthesizer 28 sets each speech segment PS sequentially generated by the mixing process (SA2) and the volume adjustment (SA3) to the pitch q1 and the pronunciation period q2 specified by the music information QM of the synthesis information Q. After adjusting (SA4), the adjusted speech elements PS are connected to each other to generate the audio signal S (SA5).

  The display control unit 24 in FIG. 1 displays various images on the display device 14. The display control unit 24 of the first embodiment causes the display device 14 to display the edited image 30 of FIG. 5 for the user to confirm and edit the contents of the synthesized music specified by the synthesis information Q. As illustrated in FIG. 5, the edited image 30 includes a score image 40 and a variable image 50.

  The score image 40 includes a piano roll type coordinate plane (hereinafter referred to as a “score region”) 42 on which a time axis (horizontal axis) and a pitch axis (vertical axis) intersect with each other, and is composed information. The contents of the synthesized music designated by the music information QM of Q are expressed. Specifically, the display control unit 24 arranges a note image 44 representing each note of the synthesized music in the score area 42. The position of the note image 44 in the direction of the pitch axis is set according to the pitch q1 specified by the music information QM, and the position and display length of the note image 44 in the direction of the time axis are pronounced specified by the music information QM. It is set according to the period q2. Further, each musical note image 44 is added with a voice code q3 designated by the music information QM. In FIG. 5, the case where the character (the lyrics of the synthesized music) and the phoneme symbol designated by the voice code q3 are arranged inside the musical note image 44 is illustrated.

  The variable image 50 is configured to include an area (hereinafter referred to as “variable area”) 52 in which a time axis (horizontal axis) is set, and a change in volume of the mixing ratio R applied to the mixing process (SA2) and volume adjustment ( The time change of the volume V applied to SA3) is expressed. The time axis of the variable area 52 is the same as the time axis of the score area 42. The specific contents of the variable image 50 will be described later.

  The instruction receiving unit 22 receives an instruction from a user according to an operation on the input device 16. For example, the user can instruct editing of the composite information Q by appropriately operating the input device 16 while confirming the edited image 30. The information management unit 26 manages the composite information Q stored in the storage device 12. Specifically, the information management unit 26 updates the composite information Q (music information QM, control information QC) according to the editing instruction received by the instruction receiving unit 22 from the user.

  FIG. 6 is a flowchart of a schematic operation of the speech synthesizer 100 according to the first embodiment. The process of FIG. 6 is started in response to an instruction from the user to the input device 16. When the process is started, the display control unit 24 causes the display device 14 to display the edited image 30 of FIG. 5 corresponding to the composite information Q stored in the storage device 12 (SB1). Then, the instruction receiving unit 22 determines whether an instruction to edit the composite information Q is received from the user (SB2).

  When the instruction receiving unit 22 receives an instruction to edit the composite information Q (SB2: YES), an editing process including an update of the edited image 30 by the display control unit 24 and an update of the composite information Q by the information management unit 26 is executed. (SB3). For example, when the addition of a note is instructed by the user, the display control unit 24 adds the note image 44 to the position instructed by the user in the score area 42, and the information management unit 26 instructs from the user. The note information (q1 to q3) thus added is added to the music information QM of the composition information Q. When movement of the existing musical note image 44 or expansion / contraction on the time axis is instructed by the user, the display control unit 24 changes the position and display length of the musical note image 44 according to the instruction from the user, and manages information. The unit 26 changes the pitch q1 and the pronunciation period q2 of the note to be edited in the music information QM according to an instruction from the user. When the change of the voice code q3 of each note is instructed by the user, the display control unit 24 changes the display of the voice code q3 of the note according to the instruction from the user, and the information management unit 26 The voice code q3 of the note in the music information QM is changed in accordance with an instruction from the user. If editing of the composite information Q is not instructed (SB2: NO), the editing process is not executed.

  When the above processing is completed, the instruction receiving unit 22 determines whether or not an instruction for voice synthesis (generation of the voice signal S) has been received from the user (SB4). When the instruction receiving unit 22 receives a voice synthesis instruction (SB4: YES), the voice synthesis unit 28 executes the voice synthesis process of FIG. 4 to which the voice library L (LA, LB) and the synthesis information Q are applied. Thus, the audio signal S is generated (SB5). On the other hand, when the voice synthesis is not instructed (SB4: NO), the voice synthesis process is not executed. In addition, the instruction receiving unit 22 determines whether or not an instruction to end the process has been received from the user (SB6). When the process end is not instructed (SB6: NO), the process transitions to step SB1, and the subsequent processes are repeated. When the process end is instructed (SB6: YES), the process of FIG. finish.

  The user can instruct editing of the temporal change of the mixing ratio R and the volume V by appropriately operating the input device 16. FIG. 7 is a flowchart of the editing process (SB3) executed by the arithmetic processing unit 10 when the instruction receiving unit 22 receives an instruction to edit the mixing ratio R and volume V over time (SB2: YES).

  When editing of the mixing ratio R and volume V is instructed, the display control unit 24 displays the adjusted image 60 on the display device 14 as illustrated in FIG. 8 (SC1). The adjustment image 60 is an image for the user to edit the time change of the mixing ratio R and the time change of the volume V within the control period of the synthesis target sound (synthesized music).

  The adjustment image 60 includes an adjustment region 62 in which a first axis A1 (horizontal axis) and a second axis A2 (vertical axis) intersecting each other are set. The first axis A1 is a coordinate axis indicating the numerical value of the mixing ratio R between the speech unit PA (first speech) and the speech unit PB (second speech), and the second axis A2 is the synthesis target speech (after mixing). Is a coordinate axis indicating the numerical value of the volume V of the speech unit PS). “Voice color A” displayed at the negative end (left end) of the first axis A1 means the voice quality of the first voice, and is displayed at the positive end (right end) of the first axis A1. “Voice color B” means the voice quality of the second voice.

  Each point in the adjustment area 62 is displayed in different display modes (characteristics of visually identifiable images such as hue, saturation, and brightness) depending on the position. Although the actual adjustment area 62 is a color image including a large number of colors, for convenience, the color image cannot be used in the patent drawing. In FIG. (Grayscale) is alternatively expressed, and the brightness (gradation) in the adjustment region 62 is alternatively expressed by the density of halftone dots on the drawing. Specifically, the distribution of hues from blue to red is expressed by the distribution of gradations from low to high gradations, and the distribution of gradations from dark to light is changed from high density (dense) to low density (sparse). It is expressed by the density of halftone dots. That is, as is understood from FIG. 8, the adjustment region 62 of the first embodiment is red (gradation: high) from the negative side (left side) end to the positive side (right side) end of the first axis A1. Changes continuously from blue to blue (gradation: low) and continuously from low gradation (halftone: dense) to high gradation (halftone: sparse) from the negative side to the positive side of the second axis A2. It is an image to be.

  The user can arbitrarily designate a plurality of time points T (T1, T2) on the time axis of the variable area 52 by appropriately operating the input device 16 while confirming the score image 40 of FIG. It is. The instruction receiving unit 22 receives instructions from a plurality of time points T (T1, T2) on the time axis from the user (SC2). The time point T1 corresponds to the start point of the control period in which the mixing ratio R and the volume V of the synthesis target sound change, and the time point T2 corresponds to the end point of the control period. As illustrated in FIG. 8, the display control unit 24 displays each time T received from the user by the instruction receiving unit 22 in the variable area 52 (SC3). While confirming the time series of the plurality of musical note images 44 of the musical score image 40 as needed, the user can select the time T1 and the time point T1 so that the portion where the mixing ratio R and volume V should be changed is included in the control period. The time T2 is indicated. The user can also move each time point T in the direction of the time axis by operating the input device 16.

  The user adjusts a plurality of points (hereinafter referred to as “instruction points”) X corresponding to the respective time points T in the variable area 52 by appropriately operating the input device 16 while confirming the adjustment image 60 of FIG. It is possible to indicate in the area 62. The instruction receiving unit 22 sequentially receives instructions from a plurality of instruction points X (X1, X2) in the adjustment area 62 from the user (SC4). As illustrated in FIG. 8, the display control unit 24 adjusts each indication point X for which the instruction receiving unit 22 has received an instruction and a path C that connects the two instruction points X that have been indicated before and after. It is displayed in the area 62 (SC5). The path C of the first embodiment is a straight line connecting the two designated points X. Note that the display of the route C may be omitted.

  One indication point X is a coordinate point corresponding to each value of the mixing ratio R and the volume V. That is, the position of the indication point X on the first axis A1 corresponds to the value of the mixing ratio R, and the position of the indication point X on the second axis A2 corresponds to the value of the volume V. The numerical value of the mixing ratio R increases as the indication point X approaches the positive end (voice tone B) of the first axis A1, and the volume V increases as the indication point X approaches the positive end of the second axis A2. The number increases.

  FIG. 9 is a schematic diagram of the adjustment region 62 in which a plurality of designated points X (X1, X2) are designated. In FIG. 9, the change of the display mode in the adjustment area 62 is omitted for convenience. The designated point X1 designated by the user corresponds to the numerical values of the mixing ratio R and the volume V at the time T1 (starting point of the control period). That is, as illustrated in FIG. 9, the numerical value r1 corresponding to the designated point X1 on the first axis A1 corresponds to the numerical value of the mixing ratio R at the time point T1, and corresponds to the designated point X1 on the second axis A2. The numerical value v1 corresponds to the numerical value of the volume V at the time point T1. On the other hand, the designated point X2 corresponds to the numerical values of the mixing ratio R and the volume V at the time point T2 (end point of the control period). That is, as illustrated in FIG. 9, the numerical value r2 corresponding to the designated point X2 on the first axis A1 corresponds to the numerical value of the mixing ratio R at the time point T2, and corresponds to the designated point X2 on the second axis A2. The numerical value v2 corresponds to the numerical value of the volume V at the time point T2. As can be understood from the above description, the indication point X1 and the indication point X2 are numerical values from the time point T1 to the time point T2 and from the time point T1 to the time point T2 and from the time point T1 to the time point T2. It expresses the time change of the volume V that continuously transitions from v1 to the numerical value v2.

  By appropriately operating the input device 16, the user selects an arbitrary designated point X (hereinafter referred to as “selected designated point X”) in the adjustment area 62, and reproduces the sound corresponding to the selected designated point X. It is possible to instruct. When the instruction receiving unit 22 receives the selection of the selection instruction point X, the voice synthesis unit 28 generates a voice signal S by voice synthesis processing using the mixing ratio R and the volume V corresponding to the selection instruction point X. Specifically, the speech synthesizer 28 is a speech segment P (PA, PB) corresponding to specific pronunciation content (for example, a character selected in advance regardless of the speech code q3 designated by the synthesis information Q). Is selected from both the audio library LA and the audio library LB (SA1), the mixing process (SA2) applying the numerical value of the mixing ratio R corresponding to the selection instruction point X, and the numerical value of the volume V corresponding to the selection instruction point X The sound signal S having a predetermined pitch and sound generation period is generated (SA4, SA5) and reproduced from the sound emitting device 18 by executing the volume adjustment (SA3) to which the above is applied. That is, the user can actually listen to the synthesized speech to which the mixing ratio R and volume V corresponding to each designated point X are applied, and generate each desired point X in the adjustment area 62 so that a desired synthesized speech is generated. Can be adjusted. For example, the volume V is set so that the change in the volume feeling of the synthesized speech due to the difference in volume during recording between the speech unit PA and the speech unit PB (change associated with the time change of the mixing ratio R) is suppressed. It is possible to adjust according to the mixing ratio R.

  As illustrated in FIG. 8, the display control unit 24 arranges the transition image 54 corresponding to each designated point X designated in the adjustment area 62 in the variable area 52 of the variable image 50 (SC6). The transition image 54 is a band-shaped image extending along the time axis, and expresses the temporal change in the mixing ratio R and volume V from the time point T1 to the time point T2. In the display control unit 24 of the first embodiment, the display mode at each time point on the time axis of the transition image 54 is the indication point in the adjustment area 62 from time T1 to time T2 on the time axis in the variable area 52. The transition image 54 is generated so as to continuously change from the display mode at X1 to the display mode at the designated point X2. That is, the display mode at the time point T1 in the transition image 54 matches the display mode at the point X1 in the adjustment area 62, and the display mode at the time point T2 in the transition image 54 is the indication in the adjustment area 62. This corresponds to the display mode at the point X2. Further, the display mode at an arbitrary time t between the time T1 and the time T2 in the transition image 54 corresponds to the time t on the path C from the designated point X1 to the designated point X2 in the adjustment region 62. It matches the display mode at the point. Therefore, the user can visually grasp the temporal change in the mixing ratio R and the volume V from the time point T1 to the time point T2 by checking the variable image 50.

  The information management unit 26 updates the control information QC of the composite information Q so that the contents of the adjustment image 60 and the variable image 50 are reflected (SC7). Specifically, the mixing ratio R and the volume V are changed along the path C from the time point T1 to the time point T2 on the time axis from the numerical value corresponding to the designated point X1 to the numerical value corresponding to the designated point X2. The control information QC is updated. That is, the information management unit 26 continuously increases the mixing ratio R from the time point T1 on the time axis to the time point T2 from the numerical value r1 corresponding to the designated point X1 to the numerical value r2 corresponding to the designated point X2 on the first axis A1. The control information QC is updated so that the volume V continuously changes on the second axis A2 from the numerical value v1 corresponding to the indication point X1 to the numerical value v2 corresponding to the indication point X2.

  As described above, in the first embodiment, the first axis A1 indicating the mixing ratio R of the speech unit PA and the speech unit PB and the second axis A2 indicating the volume V of the synthesis target speech are set. Each indication point X (X1, X2) according to an instruction from the user is set in the adjustment area 62. Then, the change over time of the mixture ratio R that transitions from the numerical value r1 corresponding to the designated point X1 on the first axis A1 to the numerical value r2 corresponding to the designated point X2, and the numerical value v1 corresponding to the designated point X1 on the second axis A2. Control information QC for designating the time change of the volume V that transitions from 1 to the numerical value v2 corresponding to the designated point X2. According to the above configuration, the user instructs the time change of the volume V in parallel with the time change instruction of the mixing ratio R while confirming the relationship between the mixing ratio R and the volume V (the time change of both). Can be instructed collectively). Therefore, there is an advantage that it is possible to reduce the work burden on the user who designates variables (mixing ratio R and volume V) applied to the speech synthesis process.

  In the first embodiment, the control information QC is generated so that the mixing ratio R and volume V transition from the numerical value at the designated point X1 to the numerical value at the designated point X2 from time T1 to time T2 on the time axis. Therefore, the user can instruct the change in the mixing ratio R and volume V over time for a specific period of the synthesis target voice (synthesized music). In addition, since the time point T1 and the time point T2 that define the control period are variably set in accordance with an instruction from the user, the time ratios of the mixing ratio R and the volume V are changed for the user's desired period of the synthesis target voice. There is also an advantage that it can be directed.

  For example, it is assumed that the volume of the second voice of voice color B (speech segment PB) is higher than that of the first voice of voice color A (speech segment PA), and the synthesis target voice is determined from the second voice (speech color B). Assume that the first sound (voice color A) is changed over time. As illustrated in FIG. 10, the position on the second axis A2 is the same between the pointing point X1 located on the positive side (voice tone B side) of the first axis A1 and the pointing point X2 located on the negative side (voice tone A side). If equal, the numerical value of the volume V is maintained substantially constant from time T1 to time T2. Therefore, in the control period in which the voice quality of the synthesized voice changes from the voice color B of the second voice to the voice color A of the first voice, the volume of the synthesized voice is controlled due to the volume difference of each voice segment P at the time of recording. Decreases over time within the period. On the other hand, when the indication point X2 is located on the positive side of the indication point X1 on the second axis A2 as illustrated in FIG. 9, the value of the volume V increases with time from the time point T1 to the time point T2. Therefore, in the control period in which the voice quality of the synthesized voice changes from the voice color B of the second voice to the voice color A of the first voice, the volume of the synthesized voice is maintained substantially constant within the control period. That is, the volume difference of each speech unit P during recording is reduced.

  In the first embodiment, each point in the adjustment region 62 is set to a different display mode, and from the time point T1 to the time point T2 on the time axis, the display mode at the point X1 in the adjustment region 62 changes from the point of display to the point X2. A transition image 54 that changes to the display mode is displayed on the display device 14. Therefore, there is also an advantage that the user can visually and intuitively grasp the temporal change of the mixing ratio R and the volume V from the time point T1 to the time point T2.

  In the above example, the two designated points X (X1, X2) in the adjustment area 62 and the two time points T (T1, T2) in the variable area 52 are exemplified, but are illustrated in FIG. As described above, it is also possible to set three or more time points T (T1, T2, T3) in the variable area 52 and three or more indication points X (X1, X2, X3) in the adjustment area 62. . The display mode of the transition image 54 arranged in the variable area 52 by the display control unit 24 is from the display mode at the designated point X1 in the adjustment area 62 to the display mode at the designated point X2 from time T1 to time T2 on the time axis. It changes continuously and from the time point T2 to the time point T3 on the time axis, it continuously changes from the display mode at the indicated point X2 in the adjustment area 62 to the display mode at the indicated point X3. In addition, the information management unit 26 changes the mixing ratio R and volume V from the time point T1 to the time point T2 along the path C12 from the numerical value at the designated point X1 to the numerical value at the designated point X2, and from the time point T2 to the time point T3. The control information QC is updated so that a transition is made along the path C23 from the value at the designated point X2 to the value at the designated point X3.

Second Embodiment
A second embodiment of the present invention will be described below. In each of the embodiments exemplified below, elements having the same functions and functions as those of the first embodiment will be referred to in the description of the first embodiment, and detailed descriptions thereof will be appropriately omitted.

  The control information QC of the composite information Q in the second embodiment specifies the time change of the second characteristic variable in addition to specifying the time change of the mixing ratio R and the first characteristic variable (volume V) as in the first embodiment. To do. Similar to the first characteristic variable, the second characteristic variable is a variable (feature amount) related to the acoustic characteristic of the synthesis target speech. In the second embodiment, the volume U is exemplified as the second characteristic variable. The volume V (first characteristic variable) and the volume U (second characteristic variable) are the same kind of acoustic characteristics, but the volume V is adjusted in consideration of the relationship with the temporal change of the mixing ratio R, whereas The volume U is adjusted in consideration of the relationship with each note of the synthesized music (time change in volume linked to the progress of the synthesized music). That is, for example, the volume V so that the change in the volume feeling of the synthesized speech due to the volume difference during recording between the speech unit PA and the speech unit PB (change associated with the temporal change in the mixing ratio R) is suppressed. Can be adjusted according to the mixing ratio R, and the volume U can be changed as a musical expression as the synthesized music progresses.

  FIG. 12 is a schematic diagram of an edited image 30 in the second embodiment. In the variable area 52 of the variable image 50 in the second embodiment, a time axis similar to that in the first embodiment and a numerical axis AY (vertical axis) intersecting the time axis are set. The numerical value axis AY is a coordinate axis indicating the numerical value of the volume U.

  In the variable image 50 of the second embodiment, the temporal change of the mixing ratio R and volume V is expressed by the display mode (hue, brightness, etc.) of the transition image 54 as in the first embodiment, and is specified by the control information QC. The time change of the volume U is expressed by the shape of the transition image 54. Specifically, the time change of the volume U is expressed by the outline 56 located at the upper edge of the transition image 54. FIG. 12 illustrates a case where the outline 56 of the transition image 54 is a broken line that expresses the change in volume U over time. A numerical value on the numerical axis AY corresponding to one point at an arbitrary time point t on the time axis in the outline 56 of the transition image 54 means a numerical value of the volume U at the time point t.

  The user can instruct editing (deformation) of the outline 56 of the transition image 54 by appropriately operating the input device 16. The display control unit 24 deforms the outline 56 of the transition image 54 according to the instruction received by the instruction receiving unit 22 from the user, and the information management unit 26 sets the volume U specified by the control information QC of the composite information Q. The time change is updated according to the instruction from the user. Specifically, the information management unit 26 updates the time change of the sound volume U to the time change represented by the outline 56 after being deformed by the display control unit 24.

  In the second embodiment, the same effect as in the first embodiment is realized. In the second embodiment, the temporal change in the mixing ratio R and the volume V is expressed by the display mode (hue, brightness, etc.) of the transition image 54, and the temporal change in the volume U in the composite music is the transition image 54. Expressed in shape. Therefore, for example, the user can confirm the time change of the sound volume U in addition to the mixing ratio R and the sound volume V with a simple display as compared with the configuration in which the time change of the sound volume U is displayed separately from the transition image 54. There is.

<Third Embodiment>
FIG. 13 is a schematic diagram of an edited image 30 in the third embodiment. In the first embodiment, the temporal change of the mixing ratio R and the volume V is expressed by the transition image 54 arranged in the variable area 52 separate from the score area 42. In the third embodiment, the time change of the mixing ratio R and the volume V is expressed using the musical note image 44 arranged in the score area 42. The contents of the adjustment image 60 are the same as in the first embodiment.

  As understood from FIG. 13, the display control unit 24 of the third embodiment is configured such that the display form of each musical note iconic image 44 in the control period is the indication point in the adjustment area 62 from the time T1 to the time T2 on the time axis. The display mode of each musical note image 44 is controlled so as to continuously change from the display mode at X1 to the display mode at the indication point X2. For example, the display mode at the time point T1 of the musical note iconic image 44 including the time point T1 among the plurality of note image images 44 in the score area 42 matches the display mode at the indication point X1 in the adjustment area 62. Similarly, the display mode at the time point T 2 in the musical note iconic image 44 of the note including the time point T 2 matches the display mode at the indication point X 2 in the adjustment area 62. Further, the display mode at the time t in the musical note image 44 of a note including an arbitrary time t on the time axis corresponds to the time t on the path C from the designated point X1 to the designated point X2 in the adjustment region 62. The display mode of the selected point is set.

  On the other hand, in the variable area 52 of the variable image 50, a time axis and a numerical value axis AY (vertical axis) are set in the same manner as in the second embodiment, and a transition line expressing the time change of the volume U specified by the control information QC. 58 is displayed. A transition line 58 in FIG. 13 is a polygonal line that represents a temporal change in the volume U, and corresponds to the outline 56 of the transition image 54 in the second embodiment. The variable image 50 can be omitted.

  In the third embodiment, the same effect as in the first embodiment is realized. Further, in the third embodiment, the time change of the mixing ratio R and the volume V is expressed according to the display mode of the musical note image 44 arranged in the score area 42. There is an advantage that the user can easily grasp the relationship with each note of the synthesized music.

<Fourth embodiment>
FIG. 14 is a schematic diagram of an edited image 30 in the fourth embodiment. As illustrated in FIG. 14, in the musical score area 42 of the fourth embodiment, a musical note image 44 representing each note of the synthesized music and an auxiliary graphic image 46 corresponding to each musical note image 44 are arranged. The corresponding musical note image 44 and auxiliary graphic image 46 are arranged at positions close to each other (that is, a position where the user can determine the correspondence between the musical note graphic image 44 and the auxiliary graphic image 46). The display length on the time axis of the auxiliary iconic image 46 is common to the musical note iconic image 44 corresponding to the auxiliary iconic image 46.

  In the third embodiment, the time change of the mixing ratio R and the sound volume V is expressed using each musical note image 44 in the musical score area 42. However, in the fourth embodiment, each auxiliary graphic image 46, which is separate from the musical note graphic image 44, is represented. This is used for expressing the temporal change in the mixing ratio R and volume V. That is, the auxiliary iconic image 46 is positioned as an iconic image for assisting display of variables (mixing ratio R, volume V) related to the note represented by the note image 44. The contents of the adjustment image 60 are the same as those in the first embodiment. Also, in the variable area 52 of the variable image 50, similarly to the second embodiment, a transition line 58 expressing the time change of the volume U specified by the control information QC is displayed. The variable image 50 can be omitted.

  As understood from FIG. 14, the display control unit 24 of the third embodiment is configured such that the display mode of each auxiliary iconic image 46 in the control period is the indication point in the adjustment area 62 from the time T1 to the time T2 on the time axis. The display mode of each auxiliary iconic image 46 is controlled so as to continuously change from the display mode at X1 to the display mode at the designated point X2. For example, the display mode of the time point T1 in the auxiliary note image 46 including the time point T1 matches the display mode at the indication point X1 in the adjustment area 62, and the time point T2 in the auxiliary icon image 46 of the note including the time point T2 is displayed. The display mode matches the display mode at the designated point X 2 in the adjustment area 62. Further, the display mode at the time t in the auxiliary graphic image 46 of the note including the arbitrary time t on the time axis corresponds to the time t on the path C from the designated point X1 to the designated point X2 in the adjustment region 62. The display mode of the selected point is set.

  In the fourth embodiment, the same effect as in the first embodiment is realized. Further, in the fourth embodiment, since the temporal change of the mixing ratio R and the volume V is expressed according to the display mode of the auxiliary graphic image 46 corresponding to each musical note graphic image 44 in the score area 42, the same as in the third exemplary embodiment. In addition, there is an advantage that the user can easily grasp the relationship between the temporal change of the mixing ratio R and the volume V and each note of the synthesized music.

  In the third embodiment in which the musical note image 44 is used to display the mixing ratio R and the volume V, the auxiliary image 46 of the fourth embodiment is unnecessary, and therefore the content of the score image 40 compared to the fourth embodiment. Has the advantage of being simplified (the total number of display elements is reduced). On the other hand, in the fourth embodiment in which the auxiliary graphic image 46 separate from the musical note image 44 is used to display the mixing ratio R and the volume V, the display mode of the musical note image 44 is independent of the temporal change of the mixing ratio R and the volume V. There is an advantage that the visibility of the voice code q3 added to each musical note graphic image 44 can be maintained, for example, which can be selected (the degree of freedom in selecting the display form of the musical note graphic image 44 is high).

<Fifth Embodiment>
In the first embodiment to the fourth embodiment, the mixing process of two types of sounds (first sound and second sound) is exemplified. However, the mixing process of mixing three or more types of speech units P having different voice qualities. Can also be envisaged. When adjusting the time variation of the mixing ratio R in a scene where three or more types of speech segments P are mixed, the user needs to determine the mixing ratio R while comprehensively considering the ratio of each voice, There is a problem that the work burden on the user is large. The fifth embodiment is a form for reducing the work burden of the user who adjusts the time change of the mixing ratio R of three or more types of sounds.

FIG. 15 is an explanatory diagram of a mixing process in the fifth embodiment. As illustrated in FIG. 15, three sound libraries L (LA, LB, LC) are stored in the storage device 12 of the fifth embodiment. The speech library LC is a set of a plurality of speech segments PC extracted from a third speech having a different voice quality (voice color) from the first speech of the speech library LA and the second speech of the speech library LB. In the mixing process (SA3) executed by the speech synthesizer 28, the speech unit PA selected from the speech library LA, the speech unit PB selected from the speech library LB, and the speech unit PC selected from the speech library LC Are mixed under a mixing ratio R. The mixing ratio R of the fifth embodiment includes the ratio (weighted value) λA of the first sound, the ratio λB of the second sound, and the ratio λC of the third sound. In the mixing process, for example, the variable pA of the speech unit PA, the variable pB of the speech unit PB, and the variable pC of the speech unit PC are mixed as R (λA, λB, This is a process of calculating a variable pS relating to the voice quality of the synthesized speech element PS by performing weighted addition according to (λC). The total value of the ratio λA, the ratio λB, and the ratio λC is 1, for example.
pS = λA · pA + λB · pB + λC · pC (b)

  As understood from the above description, the voice quality of the speech unit PS can be set to an intermediate voice quality among the speech unit PA, the speech unit PB, and the speech unit PC according to the mixing ratio R. The operation of generating the audio signal S by mutually connecting the speech segments PS sequentially generated by the mixing process on the time axis is the same as that of the first embodiment. The control information QC of the composite information Q designates the time change of the mixing ratio R (λA, λB, λC) applied to the mixing process. In the fifth embodiment, the time change of the volume V is omitted. In addition to the mathematical formula (b), a known technique can be arbitrarily employed for the mixing process of the fifth embodiment.

  FIG. 16 is a schematic diagram of an edited image 30 in the fifth embodiment. The edited image 30 of the fifth embodiment includes a score image 40 similar to that of the first embodiment, an adjustment image 60 for adjusting a temporal change in the mixing ratio R, and a variable image 50. The adjustment image 60 of the fifth embodiment includes an adjustment region 62 in which reference points G (GA, GB, GC) corresponding to each voice used for voice synthesis processing are set apart from each other. The reference point GA (voice color A) corresponds to the first voice (voice segment PA), the reference point GB (voice color B) corresponds to the second voice (voice segment PB), and the reference point GC (voice color C) is This corresponds to the third voice (speech element PC).

  Each point in the adjustment area 62 is displayed in a different display mode depending on the position. The actual adjustment area 62 is a color image including multiple colors, but in FIG. 16, for the sake of convenience, the hue in the adjustment area 62 is alternatively expressed by the shade of gray (gray scale) on the drawing. Specifically, an image obtained by arranging a continuous hue distribution over blue, blue, green, and red in order of wavelength around the center of the adjustment region 62 is suitable as the adjustment region 62.

  As in the first embodiment, the instruction receiving unit 22 receives instructions from a user at a plurality of time points T (T1, T2, T3) on the time axis of the variable area 52 (SC2), and the display control unit 24 Each time point T is displayed in the variable area 52 (SC3). Further, the instruction receiving unit 22 receives instructions from the user for a plurality of instruction points X (X1, X2, X3) in the adjustment area 62 (SC4), and the display control unit 24 receives each instruction received by the instruction receiving unit 22. The point X and the path C (C12, C23) connecting the two designated points X designated in succession are displayed in the adjustment area 62 (SC5).

  One indication point X of the fifth embodiment is a coordinate point corresponding to each numerical value of the mixing ratio R (λA, λB, λC). More specifically, the closer the designated point X is to one reference point G, the larger the ratio λ (λA, λB, λC) of the sound corresponding to the reference point G becomes in the adjustment region 62. The numerical value of each ratio λ of the mixing ratio R is determined according to the position of the instruction point X. Specific examples of the relationship between the position of the indication point X and the numerical value of each ratio λ are listed below.

FIG. 17 is a schematic diagram of the adjustment region 62 of the fifth embodiment. In FIG. 17, illustration of the difference in display mode at each point in the adjustment area 62 is omitted for convenience. The relationship between each position σ (σ1, σ2, σ3, σ4,...) Of the indicated point in FIG. 17 and the relative ratio (λA: λB: λC) of each ratio λ corresponding to the indicated point X at the position σ is For example, as follows.
(1) Position σ1 (reference point GA) λA: λB: λC = 10: 0: 0
(2) Position σ2 (reference point GB) λA: λB: λC = 0: 10: 0
(3) Position σ3 (reference point GC) λA: λB: λC = 0: 0: 0
(4) Position σ4 λA: λB: λC = 0: 10: 10
(5) Position σ5 λA: λB: λC = 5: 5: 0
(6) Position σ6 λA: λB: λC = 5: 0: 5
(7) Position σ7 λA: λB: λC = 10: 5: 5
The relationship between the indication point X and the mixing ratio R (each ratio λ) is not limited to the above examples. For example, each ratio λ is set in accordance with the position of the indication point X in the adjustment region 62 so that the distance from the reference point G to the indication point X of each audio is inversely proportional to the audio ratio λ corresponding to the reference point G. It is also possible to determine the numerical value of.

  Similar to the first embodiment, the display control unit 24 arranges the transition image 54 corresponding to each designated point X designated in the adjustment area 62 in the variable area 52 of the variable image 50 (SC6). Specifically, as understood from FIG. 16, the display mode of the transition image 54 is changed from the display mode at the designated point X1 in the adjustment area 62 from the time point T1 to the time point T2 on the time axis. While changing continuously to the display mode, from the time point T2 to the time point T3 on the time axis, it continuously changes from the display mode at the indicated point X2 in the adjustment area 62 to the display mode at the indicated point X3.

  The information management unit 26 updates the control information QC of the composite information Q so that each ratio λ of the mixing ratio R changes with time according to the numerical value at each indicated point X in the adjustment region 62 (SC7). ). Specifically, the mixing ratio R (each ratio λ) changes along the path C12 from the time point T1 to the time point T2 from the numerical value corresponding to the designated point X1 in the adjustment region 62 to the numerical value corresponding to the designated point X2. At the same time, from time T2 to time T3, the control information QC is updated so that a transition is made along the path C23 from a numerical value corresponding to the designated point X2 in the adjustment region 62 to a numerical value corresponding to the designated point X3.

  As described above, in the fifth embodiment, the adjustment area 62 in which the reference point G (GA, GB, GC) corresponding to each sound applied to the mixing process is set according to an instruction from the user. The designated point X (X1, X2, X3) is set, and the time change of the mixing ratio R (λA, λB, λC) between the designated points X is designated by the control information QC. According to the above configuration, the user can change the mixing ratio R over time while visually confirming the mutual relationship (relationship between the ratios λ of the respective sounds) of the sounds applied to the mixing process in the adjustment region 62. Can be instructed. Therefore, there is an advantage that the work burden of the user who adjusts the time change of the mixing ratio R is reduced.

  Note that, in the fifth embodiment, as in the second embodiment, a configuration in which the temporal change of the volume U specified by the control information QC is expressed by the shape of the transition image 54 (the shape of the outline 56) can be employed. Similarly to the third embodiment, a configuration in which a change in display mode between each indication point X in the adjustment area 62 is expressed according to the display mode of each musical note image 44 in the score area 42, Similarly to the embodiment, the fifth embodiment also adopts a configuration in which the change in the display mode between each indication point X in the adjustment area 62 is expressed according to the display mode of each auxiliary iconic image 46 in the score area 42. Can be done.

<Sixth Embodiment>
FIG. 18 is a schematic diagram of an adjustment image 60 that is displayed on the display device 14 by the display control unit 24 according to the sixth embodiment. The adjustment image 60 of the sixth embodiment includes an adjustment region 62 in which a first axis A1 (horizontal axis) and a second axis A2 (vertical axis) intersecting each other are set. The first axis A1 and the second axis A2 are coordinate axes indicating numerical values of different types of characteristic variables. The characteristic variable is a variable related to the acoustic characteristic of the synthesis target speech, and is applied to the speech synthesis processing by the speech synthesis unit 28. For example, speech intelligibility (brightness, clearness), breath component intensity (breathiness), male / female degree (genderfactor), pitch change (pitch-bend), volume (dynamics), pronunciation intensity (velocity) Etc. can be exemplified as characteristic variables. The first axis A1 is a coordinate axis indicating the numerical value of the first characteristic variable selected from the above examples, and the second axis A2 is a coordinate axis indicating the numerical value of the second characteristic variable different from the first characteristic variable. .

  The point that each point in the adjustment area 62 is displayed in a different display mode depending on the position, and the point that a plurality of instruction points X are set in the adjustment area 62 according to an instruction from the user is the first implementation. It is the same as the form. Similarly to the first embodiment, a plurality of time points T according to instructions from the user are set in the variable area 52 of the variable image 50, similarly to between the indicated points X in the adjustment area 62. A transition image 54 whose display mode changes between each time point T on the time axis is arranged.

  The control information QC of the composite information Q designates temporal changes of the first characteristic variable and the second characteristic variable. The information management unit 26 updates the control information QC so that the numerical values of the first characteristic variable and the second characteristic variable change over time according to the numerical values at each indicated point X in the adjustment region 62. Specifically, the numerical value of the first characteristic variable continuously transitions from the numerical value corresponding to the designated point X1 to the numerical value corresponding to the designated point X2 on the first axis A1 from the time T1 to the time T2. The control information QC is updated so that the numerical values of the two characteristic variables continuously transition from the numerical value corresponding to the designated point X1 to the numerical value corresponding to the designated point X2 on the second axis A2 from the time T1 to the time T2. The

  As described above, in the sixth embodiment, an instruction according to an instruction from the user is provided in the adjustment region 62 in which the first axis A1 of the first characteristic variable and the second axis A2 of the second characteristic variable are set. A point X (X1, X2) is set, and changes in time of the first characteristic variable and the second characteristic variable are set so as to transition from a numerical value corresponding to the designated point X1 to a numerical value corresponding to the designated point X2. According to the above configuration, the user can instruct a time change of both while confirming the relationship between the first characteristic variable and the second characteristic variable. Therefore, there is an advantage that it is possible to reduce the work load on the user who designates the characteristic variable applied to the speech synthesis process.

<Modification>
Each of the aforementioned embodiments can be variously modified. Specific modifications are exemplified below. Two or more modes arbitrarily selected from the following examples can be appropriately combined.

(1) In each form mentioned above, although each indication point X in the adjustment area | region 62 was connected with the linear path | route C, the path | route C is not limited to a straight line. For example, as illustrated in FIG. 19, a configuration in which an interpolation curve corresponding to three designated points X (X1, X2, X3) is set as a path C, or between each designated point X as illustrated in FIG. A configuration in which a curve (free curve) arbitrarily designated by the user is set as the path C can be adopted.

(2) In each of the above-described forms, the brightness and hue are made different according to the position of each point in the adjustment area 62. However, the display mode to be made different according to the position of each point in the adjustment area 62 is lightness and hue. It is not limited to. For example, as illustrated in FIG. 21, a pattern such as shading or hatching (filled pattern) can be made different according to the position of each point in the adjustment region 62. In each of the above-described embodiments, the adjustment area 62 is a color image. However, the adjustment area 62 is a black and white image, and the brightness (gradation) can be varied depending on the position of each point in the adjustment area 62. .

(3) In the first to fourth embodiments, the volume V is exemplified as the first characteristic variable adjusted together with the mixing ratio R. However, the first characteristic variable is not limited to the volume V. In the second embodiment, the volume U is exemplified as the second characteristic variable in which the time change is expressed by the shape of the transition image 54. However, the second characteristic variable is not limited to the volume U. For example, as illustrated in the sixth embodiment, the intelligibility of speech, the strength of breath components, the degree of male / female voice, the minute change in pitch, etc. It is also possible to select it as the second characteristic variable of the second embodiment.

(4) In each of the above-described embodiments, attention is paid to the time change of the variable from the time point T1 to the time point T2, but the present invention is similarly applied when adjusting the time change of the variable over the entire section of the synthesis target speech (synthesized music), for example. Can be applied to. That is, it is not essential to limit the period in which the time change of the variable is adjusted to a specific period (control period) of the synthesis target speech. Therefore, the instruction of each time point T by the user can be omitted. Moreover, the structure which sets each time T on a time-axis according to the instruction | indication from a user is not essential. Specifically, each time point T can be set for the synthesis target speech (synthesized music) by a predetermined method that does not require an instruction from the user. For example, if a singing section (for example, a phrase) of a synthesized music is detected by a known method and the starting point and ending point of the singing section are set to each time point T, the voice quality of the singing voice is made different between the front and rear of the singing section. Is possible.

(5) Although the reference point G is set at the corner of the adjustment area 62 in the fifth embodiment, the reference point G (GA, GB, GC) is set inside the adjustment area 62 as illustrated in FIG. It is also possible to do. In the example of FIG. 22, a triangular area 68 having each reference point G as a vertex is defined inside the adjustment area 62. Further, as illustrated in FIG. 22, in the configuration in which the instruction point X can be set inside and outside the region 68 defined by each reference point G, the content of the mixing process is different between the inside and outside of the region 68. It is also possible.

  For example, as illustrated in FIG. 22, it is assumed that the indication point X1 is located inside the region 68 and the indication point X2 is located outside the region 68. At each time point t corresponding to the section inside the region 68 on the route C12 in the period from the time point T1 to the time point T2, the speech synthesizer 28 performs all (three types) as in the fifth embodiment. A speech unit P (PA, PB, PC) of speech is mixed under a mixing ratio R. On the other hand, at each time point t corresponding to the section outside the area 68 on the path C12 in the period from the time point T1 to the time point T2, the sound corresponding to the two reference points G (GA, GC) close to the indication point X2. The piece P (PA, PC) is mixed under the mixing ratio R. In the example of FIG. 22, the indication point X2 and the indication point X3 are both located outside the region 68. Therefore, in the period from the time point T2 to the time point T3, the speech unit PA and the speech unit PC have the mixing ratio R. Mixed in.

(6) In the fifth embodiment, the mixing of three types of sounds (first sound, second sound, and third sound) is exemplified, but the number of types of sounds to be mixed is arbitrary, for example, four or more types It is also possible to mix the voices. FIG. 23 is a schematic diagram of the adjustment area 62 when five types of sound are mixed. As illustrated in FIG. 23, five reference points G (GA, GB, GC, GD, GE) corresponding to each voice are set on the circumference of the circular adjustment region 62.

(7) In each of the above-described embodiments, the speech synthesizer 100 that executes both management of the synthesis information Q (display control unit 24 and information management unit 26) and generation of the speech signal S (speech synthesis unit 28) is illustrated. However, the present invention can also be specified as a speech synthesis management device that manages the synthesis information Q. In the speech synthesis management device, the presence or absence of the speech synthesizer 28 is not questioned. It is also possible to realize the speech synthesizer 100 and the speech synthesis management device with a server device that communicates with a terminal device such as a mobile phone. The instruction receiving unit 22 receives an instruction given by the user to the terminal device from the terminal device via the communication network, and the display control unit 24 transmits, for example, image data of the edited image 30 to the terminal device, thereby editing the edited image 30. Is displayed on the display device of the terminal device. The voice synthesizer 28 transmits the voice signal S generated by the voice synthesis process to the terminal device.

(8) In each of the above-described embodiments, the generation of the voice signal S of the singing voice of the synthesized music has been exemplified. However, the present invention is also applied to the generation of the voice signal S of the voice other than the singing voice (for example, conversation sound). Is possible. Therefore, the music information QM (pitch q1, tone generation period q2) of the composite information Q can be omitted. In each of the above-described embodiments, Japanese speech synthesis has been exemplified, but the speech language to be synthesized is arbitrary. For example, the present invention can be applied to the case of generating speech in an arbitrary language such as English, Spanish, Chinese, or Korean.

DESCRIPTION OF SYMBOLS 100 ... Voice synthesizer, 10 ... Arithmetic processing unit, 12 ... Memory | storage device, 14 ... Display device, 16 ... Input device, 18 ... Sound emission device, 22 ... Instruction reception part, 24 ... Display Control unit 26... Information management unit 28 .. speech synthesis unit 30 .. edit image 40 .. score image 42 .. score area 44 .. note image 46 .. auxiliary image 50. Variable image 52... Variable region 54. Transition image 56. Outline line 58. Transition line 60. Adjustment image 62.

Claims (5)

An adjustment region in which a first axis indicating a mixing ratio of the first voice and the second voice at the time of synthesis of the synthesis target voice and a second axis indicating a first characteristic variable related to the acoustic characteristic of the synthesis target voice is set. Display control means for causing the display device to display the adjusted image including
An instruction receiving means for receiving an instruction of the first indicating point and the second indicating point in the adjustment area from a user;
A time change of the mixing ratio from a numerical value corresponding to the first indicating point on the first axis to a numerical value corresponding to the second indicating point, and a numerical value corresponding to the first indicating point on the second axis A speech synthesis management device comprising: information management means for generating control information indicating a time change of the first characteristic variable from a to a numerical value corresponding to the second indication point. The information management means sets the second instruction point from a numerical value corresponding to the first instruction point on the first axis for a period from the first time point to the second time point on the time axis in the synthesis target speech. A time change of the mixing ratio to a corresponding numerical value and a time change of the first characteristic variable from a numerical value corresponding to the first indicated point on the second axis to a numerical value corresponding to the second indicated point. The speech synthesis management device according to claim 1, wherein control information to be generated is generated. The display control means includes
The adjustment image in which each point in the adjustment area is set to a different display mode;
A variable image including a transition image that changes from a display mode at the first indication point in the adjustment region to a display mode at the second indication point from the first time point to the second time point on the time axis. The speech synthesis management device according to claim 2, which is displayed on a display device. The speech synthesis management device according to claim 3, wherein the display control unit causes the display device to display the variable image representing a time change of a second characteristic variable related to an acoustic property of the synthesis target speech in a shape of the transition image. The display control means includes
The adjustment image in which each point in the adjustment area is set to a different display mode;
A musical score image representing each note of the synthesis target speech and a musical score image arranged in a musical score area in which a time axis and a pitch axis are set are displayed on the display device, and on the time axis in each musical note graphic image The speech synthesis management device according to claim 2, wherein the display mode of each point is set to a display mode at a point corresponding to the point in the route from the first indication point to the second indication point in the adjustment region.