JP2009204841A - Voice processor, voice processing method, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a voice processor regenerating waveform data expressing a singing voice of the human being, and a command data expressing an accompaniment, while synchronizing therewith each other. <P>SOLUTION: A storage part 302 of the voice processor 301 stores a command sequence for regenerating a voice, and the plurality of waveform data to be regenerated synchronously, a command regeneration part 303 starts the generation of the command sequence, a lapse time measuring part 304 measures a lapse time clocked from the start of the regeneration with precision of a prescribed command time length, a fragment waveform selecting part 305 finds a breakoff to minimize an error when expressing the lapse time of breaking the waveform data to be regenerated in the near future, in each prescribed waveform time length from the head, by the command time length, and a waveform regeneration part 306 starts the regeneration of the following waveform data after the breakoff, when the lapse time comes to the lapse time of the breakoff found with the precision of the command time length. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a sound processing apparatus, a sound processing method, and a program for realizing them on a computer, which are suitable for synchronizing and reproducing waveform data representing human singing voice and the like and command data representing accompaniment thereof. About.

  Conventionally, waveform data represented by PCM (Pulse Coded Modulation), ADPCM (Adaptive Differential PCM), MP3 (MPeg audio layer-3), Vorbis, etc., MIDI (Musical Instrument Digital Interface) or MML (Music Macro Language) A technique for outputting a sound by combining a command expressed by a value given to a register of PSG (Programmable Sound Generator) and the like is used.

For example, in the following document, a PCM data recording unit that records PCM data on a portable terminal, PSG, in order to output a free and highly entertaining ringtone and reduce the burden on the user and portable terminal when updating sound source data, A PSG data recording unit for recording data is provided, and the ring tone data generation unit reads PCM data and PSG data as sound source data from the PCM data recording unit and the PSG data recording unit, and generates ring tone data by combining them. And the technique of outputting the ringtone based on this from a ringtone output speaker apparatus is disclosed.
JP 2001-245020 A

  Such speech synthesis technology can also be applied to a case where a character utterance and BGM (Back Ground Music) are synthesized and output in various games.

  Here, a technique is generally adopted in which the voice of the character is played by a voice actor and the human voice is used to eliminate unnaturalness, while the BGM uses a performance such as MIDI to suppress the amount of data. Yes.

  Therefore, in a scene where the character sings along with the accompaniment, it is necessary to synchronize the reproduction of the waveform data expressed by PCM and the reproduction of the command data expressed by MIDI.

  In general, waveform data is replayed at predetermined intervals by writing waveform data for a predetermined interval to a predetermined register or buffer, or passing a function of a predetermined reproduction library as an argument for the waveform data for the predetermined interval. It is done by doing. For example, when waveform data with a sampling rate of 44100 Hz and monaural 16 bits (2 bytes) is written at intervals of 1/60 second, which is the vertical synchronization period, 2 × 44100 × (1 / 60) = 1470 bytes will be passed. In this case, a technique for managing the waveform data in units of fragment waveforms obtained by dividing every 1470 bytes is used.

  Therefore, when compression is not performed, the elapsed time and the data size are proportional. Further, even when compression is performed as in MP3 or the like, since it is necessary to finally convert to PCM data, the same argument can be made.

  On the other hand, the command data is PSG data, MIDI data, various MML data, a command for determining the tempo of the music, a command for assigning the tone of the musical instrument to each of a plurality of channels, and at which pitch in which channel. It can be thought of as a sequence of commands that specify which sound length and sound volume to produce.

  Therefore, the time required for reproduction is not necessarily proportional to the length of the command string. Therefore, in order to execute some processing while interpreting the command string, a technique is provided for measuring the elapsed time by determining the basic precision for the note and generating an interrupt at an interval of that precision or increasing the counter. Is being used.

  In many cases, the basic accuracy is a time length obtained by dividing a beat (time length of a quarter note) by a predetermined constant. In this case, when the tempo specified by the command changes, the basic accuracy also changes.

  Now, considering the case where the character sings to accompaniment, etc., the accompaniment sound should be output without interruption, but in singing, the section called `` phrase '' where sound is generated to utter, etc., In general, there is a section called “breath” that should be virtually silent for breathing and the like. In addition, the same “phrase” may be used a plurality of times in one piece of music.

  Therefore, there is a need for a technique that can remove silent sections such as breath sections in order to reduce the amount of waveform data from data to be reproduced.

  On the other hand, when the silent section is removed, a technique for appropriately synchronizing between the reproduction of the waveform data and the reproduction of the command data is required.

  In addition, even if the silent section is not removed, the waveform data may be managed in units of measures, for example, and it may be desired to reduce the amount of data by sharing parts that are repeatedly sung in the music. Many.

  Furthermore, it is necessary to consider a situation where the measurement accuracy of the elapsed time in the command data does not match the time length of the fragment waveform in the waveform data.

  The present invention solves the above-described problems, and is a sound processing apparatus and sound processing method suitable for synchronizing and reproducing waveform data representing a human singing voice and the like and command data representing the accompaniment and the like. An object of the present invention is to provide a program that realizes these on a computer.

  In order to achieve the above object, the following invention is disclosed in accordance with the principle of the present invention.

  The speech processing apparatus according to the first aspect of the present invention includes a storage unit, a command playback unit, an elapsed time measurement unit, a fragment waveform selection unit, and a waveform playback unit, and is configured as follows.

  That is, the storage unit stores one command sequence including a command for designating a pitch and a tone length to be reproduced, designates a sound waveform having a predetermined waveform time length, and a plurality of commands to be reproduced continuously. A fragment waveform sequence composed of fragment waveforms is stored.

  Here, an elapsed time from a predetermined reference time is associated with the fragment waveform sequence, and each fragment waveform included in the fragment waveform sequence is present before the fragment waveform sequence in the fragment waveform sequence. The time obtained by multiplying the number of fragment waveforms to be multiplied by the waveform time length and adding the elapsed time associated with the fragment waveform sequence is associated as the elapsed time from the reference time.

  Typically, the command sequence is for playing accompaniment sounds and the like, and the fragment waveform sequence is for expressing human singing voices and the like. Here, each of the fragment waveform sequences corresponds to a phrase of a human singing voice (which may have a silent section at the beginning), and the phrase when the playback start of the music is used as a reference time. Is to be reproduced by the elapsed time associated with the fragment waveform sequence.

  The fragment waveform sequence is a sequence in which fragment waveforms are arranged, and the sound of the phrase is output by sequentially reproducing the fragment waveforms. Therefore, the elapsed time for each fragment waveform to be reproduced can be calculated from the elapsed time associated with the fragment waveform sequence and the number of the fragment waveform in the fragment waveform sequence (starting counting from 0). It is. Further, the time length of each fragment waveform is a time length corresponding to the buffer length at the time of waveform data reproduction.

  On the other hand, the command playback unit interprets the command sequence from the beginning, and starts playback of a sound waveform having a pitch and a tone length specified in the command sequence.

  As described above, the command specifies the tempo, the timbre of the instrument to be assigned to each channel, and which channel in which measure, which pitch, which pitch, and how loud. There are things to specify.

  Furthermore, the elapsed time measuring unit measures the elapsed time from the start of the reproduction of the audio waveform by the command reproducing unit with an accuracy of a predetermined command time length.

  As described above, the accuracy of the command time length varies depending on the function of the command playback unit, but is often the time length obtained by dividing the beat by a predetermined constant. In this case, when the tempo changes, the accuracy of the command time length also changes.

  Then, the fragment waveform selection unit selects, from among the plurality of fragment waveforms, the minimum error fragment waveform that minimizes the error when the elapsed time associated with the fragment waveform is expressed with the accuracy of the command time length. .

  In other words, the fragment waveforms included in the fragment waveform sequence are scanned in order from the top, and it is checked where reproduction starts from where the time lag between the waveform data and the command data is minimized.

The reference in this case is an error e when the elapsed time T for reproducing a certain fragment waveform is expressed by the command time length Δt at the elapsed time T. Typically, if you can find an integer n that satisfies the following condition:
Δt × n ≦ T <Δt × (n + 1)
The error e is
e = min (T-Δt × n, Δt × (n + 1) -T)
It can be expressed as

  In the present invention, it is possible to select in advance which of the plurality of fragment waveforms, from which reproduction is to be started, is less likely to cause synchronization deviation with the command sequence.

  Further, when the elapsed time measured with the accuracy of the command time length reaches the elapsed time associated with the selected error minimum fragment waveform, the waveform reproduction unit selects the selected error in the fragment waveform sequence. Start playback of fragment waveforms after the minimum fragment waveform.

  Even if the accuracy of checking the elapsed time is governed by the tempo specified in the command sequence and waveform playback is governed by intervals such as vertical synchronization interrupts, as described above, playback is performed from the fragment waveform with the smallest error. Therefore, the deviation can be suppressed as much as possible.

  Therefore, according to the present invention, the sound based on the waveform data can be reproduced starting from the minimum error fragment waveform in synchronization with the sound reproduction based on the command sequence.

  A speech processing apparatus according to another aspect of the present invention includes a storage unit, a command playback unit, an elapsed time measurement unit, a fragment waveform selection unit, and a waveform playback unit, and is configured as follows.

  That is, the storage unit stores one command sequence including a command for designating a pitch to be reproduced and a tone length, and a plurality of fragment waveform sequences composed of fragment waveforms designating a waveform having a predetermined waveform time length to be reproduced. Remembered.

  Here, for each of the plurality of fragment waveform sequences, the fragment waveform sequence is associated with an elapsed time from a predetermined reference time, and each fragment waveform included in the fragment waveform sequence is associated with the fragment waveform. The time obtained by multiplying the number of fragment waveforms existing before the fragment waveform sequence in the sequence by the waveform time length and adding the elapsed time associated with the fragment waveform sequence is associated as the elapsed time from the reference time. .

  Typically, the command sequence is for playing accompaniment sounds and the like, and the fragment waveform sequence is for expressing human singing voices and the like. Here, each of the fragment waveform sequences corresponds to a phrase of a human singing voice (which may have a silent section at the beginning), and the phrase when the playback start of the music is used as a reference time. Is to be reproduced by the elapsed time associated with the fragment waveform sequence.

  The fragment waveform sequence is a sequence in which fragment waveforms are arranged, and the sound of the phrase is output by sequentially reproducing the fragment waveforms. Therefore, the elapsed time for each fragment waveform to be reproduced can be calculated from the elapsed time associated with the fragment waveform sequence and the number of the fragment waveform in the fragment waveform sequence (starting counting from 0). It is. Further, the time length of each fragment waveform is a time length corresponding to the buffer length at the time of waveform data reproduction.

  On the other hand, the command playback unit interprets the command sequence from the beginning, and starts playback of a sound waveform having a pitch and a tone length specified in the command sequence.

  As described above, the command specifies the tempo, the timbre of the instrument to be assigned to each channel, and which channel in which measure, which pitch, which pitch, and how loud. There are things to specify.

  Furthermore, the elapsed time measuring unit measures the elapsed time from the start of the reproduction of the audio waveform by the command reproducing unit with an accuracy of a predetermined command time length.

  As described above, the accuracy of the command time length varies depending on the function of the command playback unit, but is often the time length obtained by dividing the beat by a predetermined constant. In this case, when the tempo changes, the accuracy of the command time length also changes.

  Further, the fragment waveform selection unit adds the waveform time length to the measured elapsed time for the elapsed time associated with any one of the plurality of fragment waveform sequences that is equal to or greater than the measured elapsed time. If the time is less than the time, for each of a predetermined number of fragment waveforms at the beginning of the fragment waveform sequence, the error minimum when the elapsed time associated with the fragment waveform is expressed with the accuracy of the command time length is minimized. Select a fragment waveform.

  The elapsed time associated with a fragment waveform sequence is equal to or greater than the measured elapsed time and less than the measured elapsed time plus the waveform time length. It means you have to.

  In the present invention, in such a situation, the fragment waveform included in the fragment waveform sequence is scanned sequentially from the beginning, and where reproduction starts from where the time lag between the waveform data and the command data is minimized. Find out.

The reference in this case is an error e when the elapsed time T for which the fragment waveform is to be reproduced is expressed by the command time length Δt at the elapsed time T. Typically, if you can find an integer n that satisfies the following condition:
Δt × n ≦ T <Δt × (n + 1)
The error e is
e = min (T-Δt × n, Δt × (n + 1) -T)
It can be expressed as

  The number of fragment waveforms to be scanned can be a constant, but can also be variable, and this aspect will be described later.

  When the elapsed time measured with the accuracy of the command time length reaches the elapsed time associated with the selected minimum error fragment waveform, the waveform reproduction unit includes the selected minimum error fragment waveform. In the column, the reproduction of the fragment waveform after the selected minimum error fragment waveform is started.

  Even if the accuracy of checking the elapsed time is governed by the tempo specified in the command sequence and waveform playback is governed by intervals such as vertical synchronization interrupts, as described above, playback is performed from the fragment waveform with the smallest error. Therefore, the deviation can be suppressed as much as possible.

  Therefore, according to the present invention, it is possible to reproduce the sound by the command sequence and the sound by the waveform data in an appropriately synchronized manner.

  In the audio processing device of the present invention, for each of the plurality of fragment waveform sequences, the first one or more fragment waveforms of the fragment waveform sequence are all silent fragment waveforms representing silence of the waveform time length. The fragment waveform selection unit can be configured to select the minimum error fragment waveform with the number of silent fragment waveforms at the beginning of the fragment waveform sequence as the predetermined number.

  That is, only the leading silent fragment waveform is scanned, and the fragment waveform with the smallest error is selected. Here, the silent fragment waveform is waveform data whose displacement is 0 (constant) after all. Therefore, as a format of the fragment waveform, a flag indicating whether or not it is a silent fragment waveform is prepared, and in the case of a silent fragment waveform, if the waveform data representing the displacement is omitted, the amount of data is reduced. Can be suppressed.

  According to the present invention, since the reproduction of the fragment waveform sequence is always started from the silent portion, it is possible to prevent the occurrence of a “pop” sound that is likely to occur at the start of the reproduction of the waveform data. Waveform data can be reproduced.

  The speech processing apparatus of the present invention further includes an elapsed time input receiving unit, and can be configured as follows.

  That is, the elapsed time input reception unit receives an elapsed time input that specifies an elapsed time from the reference time.

  This corresponds to the fast forward, rewind, and time warp functions prepared in a media player or the like that plays back music, and accepts designation from the user or designation by a program that controls the audio processing device. .

  Then, when the elapsed time input is accepted, the command playback unit interprets the command sequence from the beginning, and the time corresponding to the value obtained by integrating the sound length is set to the elapsed time specified in the elapsed time input. Until this time, the command is ignored, and when the elapsed time specified for the elapsed time input is reached, the reproduction of the sound waveform having the pitch and pitch specified in the command sequence is started.

  As described above, since it is necessary to interpret the command sequence from the beginning, the time length required for music reproduction is added without producing a sound, and skipping is performed until it reaches the specified elapsed time. When the specified elapsed time is reached, music playback based on command interpretation is started from there.

  On the other hand, the elapsed time measurement unit calculates the sum of the time corresponding to the value obtained by integrating the sound lengths and the elapsed time since the playback of the voice waveform by the command playback unit with the accuracy of the command time length. The measured elapsed time.

  That is, the skipped integration time is counted as the elapsed time.

  According to the present invention, it is possible to perform playback from the middle, such as fast forward, rewind, and time warp, in the synchronized playback of audio using a command sequence and waveform data.

  In the speech processing apparatus of the present invention, the elapsed time measurement unit measures the elapsed time from the start of the playback of the voice waveform by the command playback unit by measuring the number of interrupts that occur every other command time length. Then, each time the interrupt occurs, the fragment waveform selection unit generates a fragment waveform sequence whose associated elapsed time is equal to or longer than the measured elapsed time and less than the measured elapsed time plus the waveform time length. It can be configured to determine whether it is stored.

  The present invention relates to a preferred embodiment of the above invention, and it is assumed that when interpreting a command sequence, an interrupt with a predetermined period synchronized with the playback tempo of the command sequence occurs. The playback start is also controlled.

  In the audio processing device of the present invention, the waveform reproduction unit can be configured to reproduce one fragment waveform every time a vertical synchronization interrupt occurs.

  The present invention relates to a preferred embodiment of the above-described invention, and applies the present invention when performing sound processing of waveform data in synchronization with screen display processing in a game device or the like. By making the length of the audio data to be output during the period coincide with the waveform time length, the management and reproduction processing of the waveform data is facilitated.

  In the speech processing apparatus of the present invention, the command sequence is expressed by MIDI (Musical Instrument Digital Interface) or MML (Music Macro Language), and the fragment waveform is PCM (Pulse Code Modulation), ADPCM (Adaptive Differential PCM). ), MP3 (MPeg audio layer-3) or Vorbis.

  The present invention relates to a preferred embodiment of the above invention, and employs typical data as command sequence data and typical data as waveform data.

  A speech processing method according to another aspect of the present invention is executed by a speech processing apparatus having a storage unit, a command playback unit, an elapsed time measurement unit, a fragment waveform selection unit, and a waveform playback unit, and executes a command playback process and an elapsed time measurement process. And a fragment waveform selection step and a waveform reproduction step, which are configured as follows.

  That is, the storage unit stores one command sequence including a command for designating a pitch and a tone length to be reproduced, designates a sound waveform having a predetermined waveform time length, and a plurality of commands to be reproduced continuously. A fragment waveform sequence composed of fragment waveforms is stored.

  An elapsed time from a predetermined reference time is associated with the fragment waveform sequence, and each of the fragment waveforms included in the fragment waveform sequence is present before the fragment waveform sequence in the fragment waveform sequence. The time obtained by multiplying the number of fragment waveforms by the waveform time length and adding the elapsed time associated with the fragment waveform sequence is associated as the elapsed time from the reference time.

  On the other hand, in the command playback step, the command playback unit interprets the command sequence from the beginning, and starts playback of a sound waveform having a pitch and a tone length specified in the command sequence.

  Further, in the elapsed time measurement step, the elapsed time measurement unit measures the elapsed time from the start of the reproduction of the voice waveform in the command reproduction step with an accuracy of a predetermined command time length.

  Then, in the fragment waveform selection step, the error that minimizes the error when the fragment waveform selection unit expresses the elapsed time associated with the fragment waveform among the plurality of fragment waveforms with the accuracy of the command time length. Select the smallest fragment waveform.

  On the other hand, in the waveform reproduction step, when the elapsed time measured with the accuracy of the command time length reaches the elapsed time associated with the selected minimum error fragment waveform, the waveform reproduction unit The reproduction of the fragment waveforms after the selected minimum error fragment waveform is started.

  A speech processing method according to another aspect of the present invention is executed by a speech processing apparatus having a storage unit, a command playback unit, an elapsed time measurement unit, a fragment waveform selection unit, and a waveform playback unit, and executes a command playback process and an elapsed time measurement process. And a fragment waveform selection step and a waveform reproduction step, which are configured as follows.

  That is, the storage unit stores one command sequence including a command for designating a pitch to be reproduced and a tone length, and a plurality of fragment waveform sequences composed of fragment waveforms designating a waveform having a predetermined waveform time length to be reproduced. Remembered.

  For each of the plurality of fragment waveform sequences, the fragment waveform sequence is associated with an elapsed time from a predetermined reference time, and each of the fragment waveforms included in the fragment waveform sequence is associated with the fragment waveform sequence. The time obtained by multiplying the number of fragment waveforms existing before the fragment waveform sequence by the waveform time length and adding the elapsed time associated with the fragment waveform sequence is associated as the elapsed time from the reference time.

  Here, in the command playback step, the command playback unit interprets the command sequence from the beginning, and starts playback of a speech waveform having a pitch and a tone length specified in the command sequence.

  On the other hand, in the elapsed time measurement step, the elapsed time measurement unit measures the elapsed time from the start of the reproduction of the voice waveform in the command reproduction step with an accuracy of a predetermined command time length.

  Furthermore, in the fragment waveform selection step, the elapsed time associated with any one of the plurality of fragment waveform sequences is equal to or longer than the measured elapsed time, and the waveform time length is added to the measured elapsed time. If the time is less than the time, an error when the fragment waveform selection unit represents the elapsed time associated with the fragment waveform for each of the first predetermined number of fragment waveforms in the fragment waveform sequence with the accuracy of the command time length. Select the smallest error fragment waveform that minimizes.

  In the waveform reproduction step, when the elapsed time measured with the accuracy of the command time length reaches the elapsed time associated with the selected minimum error fragment waveform, the waveform reproduction unit selects the selected minimum error fragment waveform. In the fragment waveform sequence including “”, reproduction of fragment waveforms after the selected minimum error fragment waveform is started.

  A program according to another aspect of the present invention is configured to cause a computer to function as each unit of the above-described sound processing device.

  The program of the present invention can be recorded on a computer-readable information storage medium such as a compact disk, flexible disk, hard disk, magneto-optical disk, digital video disk, magnetic tape, and semiconductor memory.

  The above program can be distributed and sold via a computer communication network independently of the computer on which the program is executed. The information storage medium can be distributed and sold independently from the computer.

  ADVANTAGE OF THE INVENTION According to this invention, the audio | voice processing apparatus suitable for reproducing | regenerating synchronously the waveform data showing a human singing voice, etc., and the command data showing the accompaniment etc., and these are implement | achieved by computer A program can be provided.

  Embodiments of the present invention will be described below. In the following, for ease of understanding, an embodiment in which the present invention is realized using a game information processing device will be described. However, the embodiment described below is for explanation, and the present invention is described. It does not limit the range. Therefore, those skilled in the art can employ embodiments in which each or all of these elements are replaced with equivalent ones, and these embodiments are also included in the scope of the present invention.

  FIG. 1 is a schematic diagram showing a schematic configuration of a typical information processing apparatus that performs the function of the speech processing apparatus of the present invention by executing a program. Hereinafter, a description will be given with reference to FIG.

  The information processing apparatus 100 includes a CPU (Central Processing Unit) 101, a ROM 102, a RAM (Random Access Memory) 103, an interface 104, a controller 105, an external memory 106, an image processing unit 107, and a DVD-ROM. (Digital Versatile Disc ROM) drive 108, NIC (Network Interface Card) 109, audio processing unit 110, and microphone 111.

  A DVD-ROM storing a game program and data is loaded in the DVD-ROM drive 108 and the information processing apparatus 100 is turned on to execute the program, thereby realizing the audio processing apparatus of the present embodiment. Is done.

  The CPU 101 controls the overall operation of the information processing apparatus 100 and is connected to each component to exchange control signals and data. Further, the CPU 101 uses arithmetic operations such as addition / subtraction / multiplication / division, logical sum, logical product, etc. using an ALU (Arithmetic Logic Unit) (not shown) for a storage area called a register (not shown) that can be accessed at high speed. , Logic operations such as logical negation, bit operations such as bit sum, bit product, bit inversion, bit shift, and bit rotation can be performed. In addition, the CPU 101 itself is configured so that saturation operations such as addition / subtraction / multiplication / division for multimedia processing, vector operations such as trigonometric functions, etc. can be performed at a high speed, and those provided with a coprocessor. There is.

  The ROM 102 records an IPL (Initial Program Loader) that is executed immediately after the power is turned on, and when this is executed, the program recorded on the DVD-ROM is read out to the RAM 103 and execution by the CPU 101 is started. The The ROM 102 stores an operating system program and various data necessary for operation control of the entire information processing apparatus 100.

  The RAM 103 is for temporarily storing data and programs, and holds programs and data read from the DVD-ROM and other data necessary for game progress and chat communication. Further, the CPU 101 provides a variable area in the RAM 103 and performs an operation by directly operating the ALU on the value stored in the variable, or temporarily stores the value stored in the RAM 103 in the register. Perform operations such as performing operations on registers and writing back the operation results to memory.

  The controller 105 connected via the interface 104 receives an operation input performed when the user executes the game.

  The external memory 106 detachably connected via the interface 104 stores data indicating game play status (past results, etc.), data indicating game progress, and log of chat communication in the case of a network match ( Data) is stored in a rewritable manner. The user can record these data in the external memory 106 as appropriate by inputting an instruction via the controller 105.

  A DVD-ROM mounted on the DVD-ROM drive 108 stores a program for realizing the game and image data and audio data associated with the game. Under the control of the CPU 101, the DVD-ROM drive 108 performs a reading process on the DVD-ROM loaded therein, reads out necessary programs and data, and these are temporarily stored in the RAM 103 or the like.

  The image processing unit 107 processes the data read from the DVD-ROM by an image arithmetic processor (not shown) included in the CPU 101 or the image processing unit 107, and then processes the processed data on a frame memory ( (Not shown). The image information recorded in the frame memory is converted into a video signal at a predetermined synchronization timing and output to a monitor (not shown) connected to the image processing unit 107. Thereby, various image displays are possible.

  The image calculation processor can execute a two-dimensional image overlay calculation, a transmission calculation such as α blending, and various saturation calculations at high speed.

  Also, polygon information arranged in the virtual three-dimensional space and added with various texture information is rendered by the Z buffer method, and the polygon arranged in the virtual three-dimensional space from the predetermined viewpoint position is determined in the direction of the predetermined line of sight It is also possible to perform high-speed execution of operations for obtaining rendered images.

  Further, the CPU 101 and the image arithmetic processor operate in a coordinated manner, so that a character string can be drawn as a two-dimensional image in a frame memory or drawn on the surface of each polygon according to font information that defines the character shape. is there.

  The NIC 109 is used to connect the information processing apparatus 100 to a computer communication network (not shown) such as the Internet, and is based on the 10BASE-T / 100BASE-T standard used when configuring a LAN (Local Area Network). To connect to the Internet using an analog modem, ISDN (Integrated Services Digital Network) modem, ADSL (Asymmetric Digital Subscriber Line) modem, or cable television line. A cable modem or the like and an interface (not shown) that mediates between these and the CPU 101 are configured.

  The audio processing unit 110 converts audio data read from the DVD-ROM into an analog audio signal and outputs the analog audio signal from a speaker (not shown) connected thereto. Further, under the control of the CPU 101, sound effects and music data to be generated during the progress of the game are generated, and the corresponding sound is output from a speaker, headphones (not shown), and earphones (not shown). Output.

  When the audio data recorded on the DVD-ROM is MIDI data, the audio processing unit 110 refers to the sound source data included in the audio data and converts the MIDI data into PCM data. If the compressed audio data is in ADPCM format or Ogg Vorbis format, it is expanded and converted to PCM data. The PCM data can be output by performing D / A (Digital / Analog) conversion at a timing corresponding to the sampling frequency and outputting it to a speaker.

  Furthermore, a microphone 111 can be connected to the information processing apparatus 100 via the interface 104. In this case, the analog signal from the microphone 111 is subjected to A / D conversion at an appropriate sampling frequency so that processing such as mixing in the sound processing unit 110 can be performed as a PCM format digital signal.

  In addition, the information processing apparatus 100 uses a large-capacity external storage device such as a hard disk so as to perform the same function as the ROM 102, the RAM 103, the external memory 106, the DVD-ROM mounted on the DVD-ROM drive 108, and the like. You may comprise.

  The information processing apparatus 100 described above corresponds to a so-called “consumer video game apparatus”, but the present invention can be realized as long as it performs image processing to display a virtual space. Therefore, the present invention can be realized on various computers such as a mobile phone, a portable game device, a karaoke apparatus, and a general business computer.

  For example, a general computer, like the information processing apparatus 100, includes an image processing unit that includes a CPU, RAM, ROM, DVD-ROM drive, and NIC and has simpler functions than the information processing apparatus 100. In addition to having a hard disk as an external storage device, a flexible disk, a magneto-optical disk, a magnetic tape, and the like can be used. Further, not the controller 105 but a keyboard or a mouse is used as an input device.

(Speech processor)
FIG. 2 is an explanatory diagram showing a schematic configuration of an embodiment of a sound processing apparatus realized by causing the information processing apparatus 100 to execute a program. Hereinafter, a description will be given with reference to FIG.

  The speech processing apparatus 301 according to the present embodiment includes a storage unit 302, a command playback unit 303, an elapsed time measurement unit 304, a fragment waveform selection unit 305, a waveform playback unit 306, and an elapsed time input reception unit 307.

  The storage unit 302 stores an accompaniment voice command sequence used in a scene where a character sings a song and voice waveform data played by a voice actor.

  The command playback unit 303 plays back the voice using the above command sequence, and the elapsed time measurement unit 304 measures the elapsed time since the playback of the voice based on the command sequence is started.

  Based on the measured elapsed time, the fragment waveform selection unit 305 determines at which time point from which part of the waveform data to start playback, and the waveform playback unit 306 is determined at the determined time point. Playback of the selected part starts.

  The elapsed time input accepting unit 307 accepts the input of the start position when the audio is reproduced from the middle instead of being reproduced from the beginning, and can be omitted.

  Thus, for example, when realizing a scene in which a character sings a song, the reproduction of the singing voice played by the voice actor can be synchronized with the reproduction of the accompaniment sound by the electronic musical instrument. This will be described in detail below.

That is, the storage unit 302 stores the following information.
(1) At least one command string including a command for designating a pitch to be reproduced and a sound length. The command string is typically expressed by MIDI (Musical Instrument Digital Interface) or MML (Music Macro Language), and for example, represents an accompaniment sound reproduced in a scene where a character sings a song.
(2) A plurality of fragment waveform sequences composed of fragment waveforms specifying a waveform having a predetermined waveform time length to be reproduced. The fragment waveform is typically expressed by PCM (Pulse Code Modulation), ADPCM (Adaptive Differential PCM), MP3 (MPeg audio layer-3), or Vorbis. For example, the voice of a character (sound played by a voice actor) reproduced in a scene where the character sings a song is expressed.

  Here, each fragment waveform sequence is divided into one phrase, in the above example, for each section in which the voice produced by the voice actor is sandwiched by silence such as breathing.

  The fragment waveform sequence is composed of a sequence of fragment waveforms having a length suitable for the buffer length that is a unit of processing in the waveform reproduction unit 306, and the predetermined number at the beginning of the fragment waveform sequence is a fragment waveform of waveform data representing silence. It is typical.

  For example, consider a case where N fragment waveform sequences A [1], A [2],..., A [N] for a certain musical piece are stored in the storage unit 302.

For each of i = 1, 2,..., N, the following information is associated with the fragment waveform sequence A [i].
(A) The position of the fragment waveform sequence A [i] in the reproduced music. That is, the elapsed time T [i] at which playback of the fragment waveform sequence A [i] should start when the start of music (accompaniment) playback is set as the reference time point.
(B) The length L [i] of the fragment waveform sequence A [i], that is, the number of fragment waveforms constituting the fragment waveform sequence A [i].

  Here, if the j = 1, 2,..., L [i] th fragment waveform of the fragment waveform sequence A [i] of length L [i] is expressed as A [i] [j], it is arbitrary. The time length D required to reproduce the fragment waveform A [i] [j] is equal for i and j.

  This time length D is determined by restrictions of hardware and software constituting the waveform reproduction unit 306. For example, in a configuration in which waveform reproduction is performed using vertical synchronization that occurs every 1/60 second. 1/60 second corresponds to the time length D.

Now, the reproduction of the fragment waveform sequence A [i] starts from the elapsed time t = T [i] means the following.
(1) At the elapsed time t = T [i], playback of the fragment waveform A [i] [1] is started,
(2) At the elapsed time t = T [i] + D, start playing the fragment waveform A [i] [2]
...
(J) At the elapsed time t = T [i] + (j−1) D, the reproduction of the fragment waveform A [i] [j] is started,
...

  (L [i]) At the elapsed time t = T [i] + (L [i] -1) D, playback of the fragment waveform A [i] [L [i]] is started,

  At (L [i] +1) elapsed time t = T [i] + L [i] D, the reproduction of the fragment waveform sequence A [i] is all completed.

  That is, the elapsed time T [i] + (j−1) D is associated with the fragment waveform A [i] [j].

In addition, each of the fragment waveform sequences A [1], A [2],..., A [N] is divided into phrases by appropriately removing the breath section of a series of waveform data. Therefore, for any integer 1 ≦ i <j ≦ N:
T [i] + L [i] D ≦ T [j]

  If the waveform data displacement is expressed in B bytes and the sampling rate of the waveform data is F [Hz], the length of the fragment waveform A [i] [j] is generally B x F x D bytes. It becomes. This corresponds to the buffer length of the waveform data. By using this length as a unit, management of the waveform data can be made consistent.

Now, in order to prevent “sounds” that occur at the beginning of waveform data reproduction, it is common to place a silent fragment waveform at the beginning of the fragment waveform sequence A [i]. Therefore, in addition to the above information, the following information is associated with the fragment waveform sequence A [i].
(C) Number of silent fragment waveforms S [i] to be placed at the beginning of the fragment waveform sequence A [i].

  Then, each of the fragment waveforms A [i] [1], A [i] [2],..., A [i] [S [i]] represents a silent state and typically has a displacement of 0. Integers that mean F × D. Since this is a constant string in which only the same values are arranged, it is not necessary to store F × D columns in the storage unit 302. A constant in which F × D integers representing displacement 0 are arranged at the start of playback of the fragment waveforms A [i] [1], A [i] [2], ..., A [i] [S [i]] Generate a column. Then, this may be reproduced as waveform data.

  FIG. 3 is a flowchart showing a flow of control of audio processing executed by the audio processing device 301 according to the present embodiment. Hereinafter, a description will be given with reference to FIG. This sound processing is performed under the control of the CPU 101 of the information processing apparatus 100.

  When the audio processing is started, the CPU 101 obtains an elapsed time to start reproduction designated when starting the audio processing (step S351). The elapsed time related to the start of reproduction may be specified based on an input by the user, or may be specified as an argument of a function call in a program or the like. Therefore, the CPU 101 functions as the elapsed time input receiving unit 307 in cooperation with the RAM 103 and the like.

  Then, the command reproduction unit 303 is skipped while interpreting the command sequence stored in the storage unit 302 until the elapsed time related to the reproduction start is reached without outputting the sound (step S352).

The commands interpreted by the command playback unit 303 include the following.
(1) A command for designating the playback tempo.
(2) A command that specifies how many beats a measure has.
(3) A command for designating a tone (instrument) assigned to each channel.
(4) Start of measure.
(5) Which channel and which pitch will produce a sound within which measure.
(6) End of measure.

  In the reproduction of MIDI or the like, it is typical that the unit that can be skipped is a measure unit. In this case, the command is reproduced by scanning the commands (1), (2), (4), and (6). You can get how much time has elapsed since the start to play back.

  When the necessary number of commands are skipped from the command sequence, the elapsed time related to the skip obtained by scanning the command sequence is stored in the counter variable t (step S353), and the CPU 101 reproduces the commands in the subsequent command sequence. Is instructed to the command playback unit 303 to start (step S354).

  Thereafter, the command string reproduction process is performed in parallel with the voice process. As the command string reproduction process proceeds, the value of the counter variable t gradually increases in accordance with the elapsed time. That is, by referring to the counter variable t, it is possible to know how much time has elapsed from the start of command sequence reproduction to the present.

  Such parallel audio processing is performed by the audio processing unit 110 of the information processing apparatus 100, and the process of increasing the counter variable t is to divide one beat at the current playback tempo by a predetermined integer constant. For each specified time (hereinafter referred to as “command time length ΔT”). Such a configuration is a general sound reproduction mechanism using various PSG, MIDI, and MML.

  The value of the counter variable t increases with the accuracy of the command time length ΔT, but the command time length ΔT changes depending on the tempo of the music.

  Therefore, it is desirable that the update of the counter variable t is not incremented by 1 every time the reproduction time passes the command time length ΔT, but is incremented by the current command time length ΔT. That is, the counter variable t means the current elapsed time itself.

  Since the process of increasing the counter variable t and the process of acquiring the value of the counter variable t are performed in cooperation with the voice processing unit 110, the RAM 103, and the CPU 101, these function as the elapsed time measurement unit 304. .

Now, when the reproduction of the command sequence corresponding to the accompaniment sound is started, the elapsed time measuring unit 304 acquires the current elapsed time t (step S355), and the elapsed time T [1 associated with the fragment waveform sequence is obtained. ], T [2],..., T [N] are searched for those satisfying the following conditions (step S356).
T [i] -K × D ≦ t <T [i]

  Here, the positive constant K is a constant that determines an appropriate threshold value, and is set so that the time length required for the calculation in steps S356 to S359 is not more than K × D. For many computers, K = 1 is sufficient. Moreover, it is good also as setting the value of each S [1], S [2], ..., S [N] so that this may be materialized.

  If there is one that satisfies the above conditions (step S356; Yes), the reproduction of the fragment waveform sequence A [i] must be started soon.

  Therefore, it is associated with the first S [i] pieces of the fragment waveform sequence A [i] (in this embodiment, the number is the number of silent fragment waveforms, but can generally be a predetermined number). Errors e [1], e [2],..., E [S [i]] when the elapsed time is expressed by the current command time length Δt are calculated (step S357).

Here, in calculating the error, a truncation operation (floor operation) floor (•) for truncating the decimal part of the argument is considered. The truncation operation floor (x) is
x ≦ n <x + 1
Returns an integer value n that satisfies.

Then, a function e (T, Δt) representing an error when a certain time T is expressed with the accuracy of the command time length Δt is:
e (T, Δt) = min (T-Δt × floor (T / Δt), Δt × (floor (T / Δt) +1) -T)
It can be calculated as follows.

In addition, the following various error functions may be used.
e (T, Δt) = T−Δt × floor (T / Δt);
e (T, Δt) = Δt × (floor (T / Δt) +1) -T

  In addition to the rounding operation floor (•), a rounding operation (ceiling operation) ceil (•), a rounding operation (rounding operation) round (•), or the like may be used.

If the error function is determined as described above, the error for the first S [i] fragment waveforms can be calculated as follows.
(1) e [1] = e (T [i], Δt);
(2) e [2] = e (T [i] + D, Δt);
…;
(J) e [j] = e (T [j] + (j−1) D, Δt);
…;

  (S [i]) e [S [i]] = e (T [i] + (S [i] -1) D, Δt)

  Then, the error e [1], e [2],..., E [S [i]] having the smallest value is selected (step S358).

  FIG. 4 is an explanatory diagram showing the concept of fragment waveform error for a certain fragment waveform sequence A [i]. Hereinafter, a description will be given with reference to FIG.

  In the example shown in this figure, a time axis 401 is set as the horizontal axis. In addition, the time axis 401 is divided for each command time length Δt.

  In this example, for the fragment waveform sequence A [i] (original fragment waveform sequence 402), S [i] = 4, and the fragment waveform sequence A [i] (original fragment waveform sequence 402). , A [i] [1], A [i] [2], ..., A [i] [4] (shown in white rectangles in the figure) The sound fragment waveforms A [i] [5], A [i] [6],... (In the figure, they are drawn with hatched rectangles) are continued.

  Comparing the error e [1], e [2], ..., e [4] when the start time of the silent fragment waveform is measured in units of Δt, in the example shown in this figure, e [3] Is the smallest.

  Therefore, from the re-neighboring section 403 of A [i] [3], a part of the fragment waveform sequence A [i] [3], A [i] [4], A [i] [5],. [i] [L [i]] (reproduced fragment waveform sequence 404) starts to be reproduced.

  A time lag e [3] will occur between the original fragment waveform sequence 402 and the reproduced fragment waveform sequence 404. This is considered inevitable in synchronous reproduction. According to the method, the time lag can be minimized.

  In addition, since the fragment waveform sequence 404 to be reproduced is obtained by removing some of the silent portions at the beginning of the original fragment waveform sequence 402, even if the fragment waveform sequence 404 is reproduced from the middle, the listener will not receive an unnatural impression. Absent.

  On the other hand, if a predetermined number S [i] is determined by arranging sounded fragment waveforms without arranging silent fragment waveforms at the beginning of the fragment waveform sequence, the sound based on the waveform data is suddenly started from the middle. However, depending on the application, such a simple method may be adopted.

  Also, if there are errors with the same error magnitude, the error that is as close to the head as possible is selected. In general, it is assumed that the k-th error e [k] of the first fragment waveform is the smallest.

  Then, in reproducing the fragment waveform sequence A [i], the fragment waveforms A [i] [k], A [i] [k + 1],... From the time T [k] + (k−1) D If A [i] [L [i]] are reproduced in order, synchronization with command reproduction is possible with a minimum error.

Returning to FIG. 3, the elapsed time t measured by the elapsed time measuring unit 304 is acquired (step S359), and the t is
T [k] + (k-1) D≤t
That is, whether or not the elapsed time t measured with the accuracy of the command time length Δt has reached the elapsed time T [k] + (k−1) D at which playback of the selected fragment waveform should be started (Step S360), and until that happens (step S360; No), the process of returning to step S359 is repeated.

  On the other hand, when the time T [k] + (k−1) D at which reproduction of the fragment waveform A [i] [k] should be started is reached (step S360; Yes), the CPU 101 determines that the fragment waveform A [i]. An instruction is given to the audio processing unit 110 to sequentially reproduce [k], A [i] [k + 1],..., A [i] [L [i]] (step S361), and step S355. Return to.

  In step S360, the comparison with the inequality sign (≦), not the comparison with the equal sign (=), is performed because the tempo changes during the repetition of steps S359 to S360, and the command time length changes. This is to cope with extremely exceptional situations.

Therefore, in a general situation, when the condition in step S360 is satisfied,
T [k] + (k-1) D = t
You can think that

  On the other hand, if there is no fragment waveform sequence to start reproduction (step S356; No), the process may return to step S355 as it is.

  In the above processing, the repeated portion that executes some processing after obtaining the elapsed time t may be implemented by the interrupt handler of the voice processing unit 110 that implements the elapsed time measuring unit 304.

  As described above, according to the present embodiment, it is possible to reproduce the sound by synchronizing the reproduction of the waveform data with the reproduction by the command sequence while using the time control unit of the audio processing by the command sequence.

  In this embodiment, the elapsed time input receiving unit 307 and steps S351 to S353 are provided in order to enable playback from the middle. However, these configurations are omitted, and the counter variable t is initialized to 0. It is also possible to adopt a form in which step S354 and subsequent steps are executed after conversion.

  When this embodiment is applied to a situation where a character sings, the character's singing voice is common, and if multiple MIDI or MML playing accompaniment sounds are prepared, one of them can be selected. A certain singing voice can be reproduced with various accompaniment sounds.

  As described above, according to the present invention, a sound processing device, a sound processing method, and a sound processing apparatus suitable for synchronizing and reproducing waveform data representing a human singing voice and the like and command data representing the accompaniment, etc. A program for realizing these by a computer can be provided.

It is a schematic diagram which shows the outline | summary structure of the typical information processing apparatus which performs the function of the audio | voice processing apparatus of this invention by running a program. It is explanatory drawing which shows the schematic structure of one Embodiment of the audio processing apparatus implement | achieved by making the said information processing apparatus run a program. It is a flowchart which shows the flow of control of the audio | voice process performed with the audio | voice processing apparatus which concerns on this embodiment. It is explanatory drawing which shows the concept of the error of the fragment waveform about a certain fragment waveform sequence A [i].

Explanation of symbols

100 Information processing apparatus 101 CPU
102 ROM
103 RAM
104 Interface 105 Controller 106 External Memory 107 Image Processing Unit 108 DVD-ROM Drive 109 NIC
DESCRIPTION OF SYMBOLS 110 Voice processing part 111 Microphone 301 Voice processing apparatus 302 Storage part 303 Command reproduction part 304 Elapsed time measurement part 305 Fragment waveform selection part 306 Waveform reproduction part 307 Elapsed time input reception part 401 Time axis 402 Original fragment waveform sequence 403 Minimal error The nearest segmentation of the fragment waveform (at the start of playback)
404 Fragment waveform sequence to be reproduced

Claims (11)

One command sequence including a command for designating the pitch and tone length to be reproduced is stored, a waveform waveform of a predetermined waveform time length is designated, and a fragment waveform sequence comprising a plurality of fragment waveforms to be reproduced continuously Including a storage unit for storing
The fragment waveform sequence is associated with an elapsed time from a predetermined reference time, and each fragment waveform included in the fragment waveform sequence has a fragment waveform existing before the fragment waveform sequence in the fragment waveform sequence. The time obtained by multiplying the number of the waveform times by the waveform time length and adding the elapsed time associated with the fragment waveform sequence is associated as the elapsed time from the reference time,
A command playback unit that interprets the command sequence from the beginning and starts playback of a sound waveform having a pitch and a pitch specified in the command sequence;
An elapsed time measuring unit that measures the elapsed time from the start of the reproduction of the audio waveform by the command reproducing unit with an accuracy of a predetermined command time length;
Of the plurality of fragment waveforms, a fragment waveform selection unit that selects an error minimum fragment waveform that minimizes an error when the elapsed time associated with the fragment waveform is expressed with the accuracy of the command time length,
When the elapsed time measured with the accuracy of the command time length reaches the elapsed time associated with the selected minimum error fragment waveform, the fragments after the selected minimum error fragment waveform in the fragment waveform sequence An audio processing apparatus comprising: a waveform reproduction unit that starts waveform reproduction. There is provided a storage unit for storing one command sequence including a command for designating a pitch to be reproduced and a tone length, and storing a plurality of fragment waveform sequences composed of fragment waveforms for designating a waveform having a predetermined waveform time length to be reproduced. ,
For each of the plurality of fragment waveform sequences, the fragment waveform sequence is associated with an elapsed time from a predetermined reference time, and each of the fragment waveforms included in the fragment waveform sequence is associated with the fragment waveform sequence. The time obtained by multiplying the number of fragment waveforms existing before the fragment waveform sequence by the waveform time length and adding the elapsed time associated with the fragment waveform sequence is associated as the elapsed time from the reference time,
A command playback unit that interprets the command sequence from the beginning and starts playback of a sound waveform having a pitch and a pitch specified in the command sequence;
An elapsed time measuring unit that measures the elapsed time from the start of the reproduction of the audio waveform by the command reproducing unit with an accuracy of a predetermined command time length;
When the elapsed time associated with any one of the plurality of fragment waveform sequences is equal to or longer than the measured elapsed time and less than the time obtained by adding the waveform time length to the measured elapsed time, For each of a predetermined number of fragment waveforms at the beginning of the fragment waveform sequence, a fragment that selects the minimum error fragment waveform that minimizes the error when the elapsed time associated with the fragment waveform is expressed with the accuracy of the command time length. Waveform selector,
When the elapsed time measured with the accuracy of the command time length reaches the elapsed time associated with the selected minimum error fragment waveform, among the fragment waveform sequence including the selected minimum error fragment waveform, An audio processing apparatus comprising: a waveform reproduction unit that starts reproduction of fragment waveforms after the selected minimum error fragment waveform. The speech processing apparatus according to claim 2,
For each of the plurality of fragment waveform sequences, the first one or more fragment waveforms of the fragment waveform sequence are all silent fragment waveforms representing silence of the waveform time length,
The speech processing apparatus, wherein the fragment waveform selection unit selects a minimum error fragment waveform with the number of silent fragment waveforms at the head of the fragment waveform sequence as the predetermined number. The speech processing apparatus according to claim 2 or 3,
An elapsed time input receiving unit that receives an elapsed time input that specifies an elapsed time from the reference time;
When the elapsed time input is accepted, the command playback unit interprets the command sequence from the beginning, and the time corresponding to the value obtained by integrating the sound length reaches the elapsed time specified in the elapsed time input. Until this command is ignored and the elapsed time specified for the elapsed time input is reached, playback of the sound waveform of the pitch and pitch specified in the command sequence is started,
The elapsed time measuring unit calculates a sum of a time corresponding to a value obtained by integrating the sound lengths and an elapsed time since the reproduction of the voice waveform by the command reproduction unit is started with the accuracy of the command time length. A voice processing device characterized by having measured elapsed time. The speech processing apparatus according to any one of claims 2 to 4,
The elapsed time measurement unit measures the elapsed time from the start of the reproduction of the voice waveform by the command reproduction unit by measuring the number of interruptions that occur every other command time length,
The fragment waveform selection unit, each time the interrupt occurs, the associated elapsed time is not less than the measured elapsed time and less than a time obtained by adding the waveform time length to the measured elapsed time. It is judged whether it is memorize | stored. The audio processing apparatus characterized by the above-mentioned. The speech processing apparatus according to any one of claims 2 to 5,
The waveform processing unit reproduces one fragment waveform every time a vertical synchronization interrupt occurs. The speech processing apparatus according to any one of claims 1 to 6,
The command sequence is expressed by MIDI (Musical Instrument Digital Interface) or MML (Music Macro Language),
An audio processing apparatus characterized in that the fragment waveform is expressed by PCM (Pulse Code Modulation), ADPCM (Adaptive Differential PCM), MP3 (MPeg audio layer-3) or Vorbis. A speech processing method executed by a speech processing apparatus having a storage unit, a command playback unit, an elapsed time measurement unit, a fragment waveform selection unit, and a waveform playback unit,
The storage unit stores one command sequence including a command for designating a pitch to be reproduced and a tone length, designates a waveform of a sound having a predetermined waveform time length, and a plurality of fragments to be reproduced continuously. A fragment waveform sequence consisting of waveforms is stored,
The fragment waveform sequence is associated with an elapsed time from a predetermined reference time, and each fragment waveform included in the fragment waveform sequence has a fragment waveform existing before the fragment waveform sequence in the fragment waveform sequence. The time obtained by multiplying the number of the waveform times by the waveform time length and adding the elapsed time associated with the fragment waveform sequence is associated as the elapsed time from the reference time,
A command playback step in which the command playback unit interprets the command sequence from the beginning and starts playback of a sound waveform having a pitch and a pitch specified in the command sequence;
An elapsed time measuring step in which the elapsed time measuring unit measures an elapsed time from the start of reproduction of a voice waveform in the command reproduction step with a precision of a predetermined command time length;
The fragment in which the fragment waveform selection unit selects the minimum error fragment waveform that minimizes the error when the elapsed time associated with the fragment waveform is expressed with the accuracy of the command time length among the plurality of fragment waveforms. Waveform selection process,
When the elapsed time measured with the accuracy of the command time length reaches the elapsed time associated with the selected minimum error fragment waveform, the waveform reproduction unit selects the selected error in the fragment waveform sequence. An audio processing method comprising: a waveform reproduction step of starting reproduction of a fragment waveform after the minimum fragment waveform. A speech processing method executed by a speech processing apparatus having a storage unit, a command playback unit, an elapsed time measurement unit, a fragment waveform selection unit, and a waveform playback unit,
The storage unit stores one command sequence including a command for designating a pitch to be reproduced and a tone length, and stores a plurality of fragment waveform sequences composed of fragment waveforms designating a waveform having a predetermined waveform time length to be reproduced. And
For each of the plurality of fragment waveform sequences, the fragment waveform sequence is associated with an elapsed time from a predetermined reference time, and each of the fragment waveforms included in the fragment waveform sequence is associated with the fragment waveform sequence. The time obtained by multiplying the number of fragment waveforms existing before the fragment waveform sequence by the waveform time length and adding the elapsed time associated with the fragment waveform sequence is associated as the elapsed time from the reference time,
A command playback step in which the command playback unit interprets the command sequence from the beginning and starts playback of a sound waveform having a pitch and a pitch specified in the command sequence;
An elapsed time measuring step in which the elapsed time measuring unit measures an elapsed time from the start of reproduction of a voice waveform in the command reproduction step with an accuracy of a predetermined command time length;
When the elapsed time associated with any one of the plurality of fragment waveform sequences is equal to or longer than the measured elapsed time and less than the time obtained by adding the waveform time length to the measured elapsed time, An error that minimizes an error when the fragment waveform selection unit represents the elapsed time associated with the fragment waveform with the accuracy of the command time length for each of the first predetermined number of fragment waveforms in the fragment waveform sequence. Fragment waveform selection step for selecting the minimum fragment waveform,
When the elapsed time measured with the accuracy of the command time length reaches the elapsed time associated with the selected minimum error fragment waveform, the waveform reproduction unit includes the fragment including the selected minimum error fragment waveform. An audio processing method comprising: a waveform reproduction step of starting reproduction of a fragment waveform after the selected minimum error fragment waveform in the waveform sequence. Computer
One command sequence including a command for designating the pitch and tone length to be reproduced is stored, a waveform waveform of a predetermined waveform time length is designated, and a fragment waveform sequence comprising a plurality of fragment waveforms to be reproduced continuously Function as a storage unit that stores
The fragment waveform sequence is associated with an elapsed time from a predetermined reference time, and each fragment waveform included in the fragment waveform sequence has a fragment waveform existing before the fragment waveform sequence in the fragment waveform sequence. The time obtained by multiplying the number of times by the waveform time length and adding the elapsed time associated with the fragment waveform sequence is made to function as the elapsed time from the reference time,
A command playback unit that interprets the command sequence from the beginning and starts playback of a sound waveform having a pitch and a pitch specified in the command sequence;
An elapsed time measuring unit that measures the elapsed time from the start of the reproduction of the audio waveform by the command reproducing unit with an accuracy of a predetermined command time length;
Of the plurality of fragment waveforms, a fragment waveform selection unit that selects an error minimum fragment waveform that minimizes an error when the elapsed time associated with the fragment waveform is expressed with the accuracy of the command time length,
When the elapsed time measured with the accuracy of the command time length reaches the elapsed time associated with the selected minimum error fragment waveform, fragments after the selected minimum error fragment waveform in the fragment waveform sequence A program that further functions as a waveform playback unit that starts waveform playback. Computer
Function as a storage unit that stores one command sequence including a command that specifies a pitch and a tone length to be reproduced, and that stores a plurality of fragment waveform sequences that consist of fragment waveforms that specify a waveform having a predetermined waveform time length to be reproduced Let
For each of the plurality of fragment waveform sequences, the fragment waveform sequence is associated with an elapsed time from a predetermined reference time, and each of the fragment waveforms included in the fragment waveform sequence is associated with the fragment waveform sequence. Function that the time obtained by multiplying the number of fragment waveforms existing before the fragment waveform sequence by the waveform time length and adding the elapsed time associated with the fragment waveform sequence is correlated as the elapsed time from the reference time Let
A command playback unit that interprets the command sequence from the beginning and starts playback of a sound waveform having a pitch and a pitch specified in the command sequence;
An elapsed time measuring unit that measures the elapsed time from the start of the reproduction of the audio waveform by the command reproducing unit with an accuracy of a predetermined command time length;
When the elapsed time associated with any one of the plurality of fragment waveform sequences is equal to or longer than the measured elapsed time and less than the time obtained by adding the waveform time length to the measured elapsed time, For each of a predetermined number of fragment waveforms at the beginning of the fragment waveform sequence, a fragment that selects the minimum error fragment waveform that minimizes the error when the elapsed time associated with the fragment waveform is expressed with the accuracy of the command time length. Waveform selector,
When the elapsed time measured with the accuracy of the command time length reaches the elapsed time associated with the selected minimum error fragment waveform, among the fragment waveform sequence including the selected minimum error fragment waveform, A program which further functions as a waveform reproduction unit for starting reproduction of fragment waveforms after the selected minimum error fragment waveform.

Images