JPH10319955A - Voice data processor and medium recording data processing program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To make it possible to process text data including KANA (Japanese syllabary), etc., by retrieving plural kinds of characters as the delimition data indicating the delimition of the character strings from the character strings included in text data, allocating the character strings delimited by the retrieved characters to the data of one note in note data to from voice data. SOLUTION: Import processing allocates the text data 52 to playing data 51 and forms note plus data 53. The playing data 51 has note events N1 to N8 corresponding to eight notes. The text data 52 is broken down to text events according to text delimition signals and is allocated to the respective note events N1 to N8. The note plus data 53 deletes KANJI (Chinese characters) by taking read KANA and rubies into consideration and leaves only the KANA corresponding thereto. Namely, the processing of the text data 52 including the KANA or KANJI, etc., is made possible and, therefore, the input or edition of the text data 52 is simple.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a voice data processing technique, and more particularly to a voice data processing technique for causing a voice generating device to generate a voice.

[0002]

2. Description of the Related Art There is known a voice data processing device capable of processing voice data supplied to a voice generating device. The speech sounding device includes a speech synthesis circuit such as a formant sound source. The formant sound source generates an audio signal by synthesizing a formant formed by frequency-analyzing the audio.

[0003] A syllable is one of the so-called 50 sounds, for example, "ka". Syllables can be broken down into phoneme sequences on the time axis. For example, the syllable “ka” is obtained by the phoneme sequence “CL (7.5 ms) + kha (4 × 7.5 ms) +
aj (infinite length) ". The phonemes are, for example, “CL”, “kha”, and “aj”. The formant sound source generates an audio signal by inputting a phoneme sequence.

[0004] Japanese Patent Application Laid-Open Nos. 9-50287 and 9-4417 disclose conventional audio data processing devices.
No. 9 is known. These store melody data and lyrics data in a memory, sequentially read out both data from the memory, and supply them to a formant sound source. The formant sound source receives the data and generates a singing voice.

[0005] As a data processing device, a word processor capable of editing syllables is known. However, since there is no data processing device capable of editing phonemes, voice data has been manually set.

[0006]

Problems to be Solved by the Invention Japanese Patent Laid-Open No. 9-50287
In the data processor of the publication, it is necessary to input the lyrics data in Roman alphabet notation, and cannot use kana or kanji, so that input or editing of the lyrics data is inconvenient.
For example, it is not possible to use existing lyrics data in which kana or kanji are mixed.

Also, in order to assign lyrics data to each note, it is necessary to provide a note delimiter (lyric delimiter) symbol in the lyrics data, but the lyrics delimiter is limited to only one symbol (for example, space "_"). ing. Therefore, a line feed mark (a mark indicating the position of a line feed on the lyrics display screen) and a sub-line feed mark (usually not used, but the lyrics up to the line feed mark are displayed on one line due to the display device such as a small display space. In the case of karaoke lyrics data including two types of line feed marks (a mark indicating the position of a line feed before the line feed mark when the line feed is not possible), it is possible to use only the line feed mark as a lyrics delimiter. However, both the line feed mark and the sub-line feed mark cannot be used as lyrics delimiters.

Further, the lyrics data and the like have a unique data format, and a dedicated device is required as a playback device for the lyrics data, and cannot be reproduced by a general sequencer or the like.

An object of the present invention is to provide a voice data processing device or a computer program recording medium capable of processing text data including kana and the like.

Another object of the present invention is to provide an audio data processing device or a recording medium for a computer program which can easily edit audio data.

It is another object of the present invention to provide an audio data processing device or a computer program recording medium capable of processing general-purpose audio data.

[0012]

According to one aspect of the present invention, storage means for storing text data including a character string and note data including data in note units, and storing a plurality of types of characters in a character string As delimiter data indicating the delimiter of
Generating means for searching a character string included in the text data and assigning a character string delimited by the searched character to data of one note in the note data to generate audio data; A data processing device is provided.

According to another aspect of the present invention, a storage means for storing text data including a character string and note data including data in units of notes, and a plurality of types of characters as exhalation symbols, Generating means for searching from among character strings included therein, adding an exhalation mark to a character string delimited by the searched character, and assigning it to data of one note in the note data to generate voice data. Data processing device having the same. Is provided.

According to another aspect of the present invention, storage means for storing text data including data in which a non-kana and its reading kana or ruby are paired, and note data including data in note units, There is provided an audio data processing apparatus having a generating unit for generating audio data by discarding non-kana in data and assigning only the reading kana or ruby to note data in the note data.

According to another aspect of the present invention, a storage means for storing audio data in which text data and note data are paired, and a change for writing by changing the combination of the text data and note data And an audio data processing device having means.

According to another aspect of the present invention, storage means for storing text data including a character string composed of a plurality of characters and note data including data in note units, and storing one character in the text data in the note Generating means for generating audio data by allocating the data to one note in the data.

According to another aspect of the present invention, first data in which first character string data and first note data are paired, followed by second character string data and second note data Storage means for storing a second set of data,
Designating means for designating a division position in the first character string data; and dividing the first character string data into two at the division position, and converting the rear character string data into the second character string data. An audio data processing device having a merging processing means is provided.

According to another aspect of the present invention, a storage means for storing voice data in which character data and note data are paired, and when the character data is a kanji, an alphanumeric character or a symbol, the character data is stored. Speech having presentation means for presenting kana candidates for pronunciation and changing means for designating one kana from the kana candidates, changing the character data to the kana, and storing the data in the storage means. A data processing device is provided.

According to another aspect of the present invention, a storage means for storing voice data in a form in which lyric data and note data are paired, and a pair with the note data a minute before the note data. So that the lyrics data is located
There is provided an audio data processing apparatus having recording means for converting the audio data and recording the lyrics data and the note data as audio data in an independent format.

According to another aspect of the present invention, the lyrics data and the note data are audio data in independent formats,
Storage means for storing audio data in which the lyrics data paired with the note data is located before the minute timing of the note data, reading the audio data from the storage means,
There is provided an audio data processing apparatus having a generation unit for generating audio data in a format in which the lyrics data and the note data are combined.

[0021]

FIG. 1 is a diagram showing a connection between a personal computer 1 and an external tone generator 21. As shown in FIG. The personal computer 1 includes the audio data processing device according to the present embodiment. The personal computer 1 may be a sequencer.

First, the configuration of the external sound source device 21 will be described. The detection circuit 33 detects an operation of the switch 34 and generates a switch signal. The switch 34 includes, for example, a switch for setting various parameters.

The bus 22 includes a detection circuit 33 and a RAM
24, ROM 25, CPU 26, display circuit 28, MID
The I interface 23, the voice synthesis circuit 32, and the musical tone waveform synthesis circuit 29 are connected.

The ROM 25 stores formant data for synthesizing speech, various other data, and computer programs. The RAM 24 stores flags, buffers, and the like. The computer program is ROM2
5 may be stored in the RAM 24. The CPU 26 performs calculation or control according to a computer program stored in the ROM 25 or the RAM 24.

The CPU 26 generates musical tone parameters based on performance data received from the personal computer 1 via the MIDI interface 23 and supplies the musical tone parameters to the musical tone waveform synthesis circuit 29. The musical sound waveform synthesis circuit 29 generates a musical sound signal according to the supplied musical sound parameter, and supplies it to the sound system 31. Sound system 31
Includes a D / A converter and a speaker, converts a supplied digital tone signal into an analog format, and generates a sound.

The tone waveform synthesizer 29 includes a waveform memory system, an FM system, a physical model system, a harmonic synthesis system,
Any method such as a formant synthesis method or an analog synthesizer method of VCO + VCF + VCA may be used.

The MIDI interface 23 is connected to the MIDI interface 8 of the personal computer 1 by M.
Connected by IDI cable. The external sound source device 21 and the personal computer 1 can perform MIDI communication.

The CPU 26 receives audio data from the personal computer 1 via the MIDI interface 23 and stores it in the RAM 24. The audio data is
Includes note data and lyrics data.

The CPU 26 reads out audio data stored in the RAM 24 and supplies formant data, note number (pitch) and the like to the audio synthesizing circuit 32 based on the formant data and the like stored in the ROM 25. The formant data is, for example, formant center frequency data, formant bandwidth data, formant level data, and the like.

The voice synthesizing circuit 32 generates a voice signal according to the supplied formant data and the like. The audio signal has a predetermined pitch and corresponds to a singing voice. The voice synthesis circuit 32 may use a formant synthesis method (formant sound source) or another method. If the method of the voice synthesis circuit 32 is other than the formant synthesis method, the data to be supplied is not formant data but data specific to the voice synthesis method.

The case where the speech synthesis circuit 32 is a formant sound source will be described. The formant data and the like generated according to the sound data are input to the sound synthesis circuit 32. The formant data includes formant center frequency data, formant level data, formant bandwidth data, and the like.

The voice synthesis circuit 32 has a voiced voice synthesis group VTG and an unvoiced voice synthesis group UTG. The voiced sound group VTG has a first group according to the formant data or the like.
To 4 voiced sound formants V1, V2, V3, and V4, respectively. The unvoiced sound synthesis group UTG, according to the formant data,
It has unvoiced sound synthesizers U1, U2, U3, and U4 for forming first to fourth unvoiced sound formants, respectively.

The first formant generation unit TG1 has a voiced sound synthesis unit V1 and an unvoiced sound synthesis unit U1 for forming a first formant, and the second formant generation unit TG2 has a voiced sound synthesis for forming a second formant. The third formant generation unit TG3 includes a voiced sound synthesis unit V3 and an unvoiced sound synthesis unit U3 for forming a third formant, and the fourth formant generation unit TG4 includes a fourth formant generation unit TG4. Has a voiced sound synthesizer V4 and an unvoiced sound synthesizer U4.

The synthesizing units V1 to V4 and U1 to U4 output audio signals corresponding to the generated formants. These audio signals are added, and the sound system 3
1 is supplied. The sound system 31 converts the supplied digital audio signal into an analog audio signal and emits the sound.

A more specific configuration of the formant sound source is as follows.
For example, it is shown in FIG. 1 of JP-A-3-200299.

The voice synthesizing circuit 32 is not limited to having four formant generation units TG1 to TG4. It may have more or less systems of formant generation units.

For a special sound such as a popping sound, a dedicated unit may be provided. Alternatively, a plurality of pronunciation channels may be prepared so that a singing of two or more voices can be generated. In this case, a plurality of tone generation channels may be formed by using one circuit in a time-division manner, or a type in which one tone generation channel is constituted by one circuit may be used. .

Speech synthesis circuit 32 and tone wave synthesis circuit 29
Is not limited to a configuration using dedicated hardware, but may be configured using a DSP + microprogram, or may be configured using a CPU + software program.

Next, the configuration of the personal computer 1 will be described. The detection circuit 11 detects an input of a key (numerical key, character key, or the like) on the keyboard 12 and generates a key signal. The detection circuit 9 detects a moving operation of the mouse 10 and an operation of a switch (right button, left button, etc.) and generates a mouse signal. The operator can use the mouse 10 or the keyboard 12
Can be used to edit audio data.

The display circuit 7 can display an editing screen for editing audio data and the like. The operator can edit the audio data while referring to the editing screen on the display circuit 7.

The bus 2 includes a detection circuit 11, a detection circuit 9, a display circuit 7, a MIDI interface 8, an RA
M3, ROM 4, CPU 5, external storage device 13, and communication interface 14 are connected.

The ROM 4 stores various parameters and computer programs. The RAM 3 stores flags, buffers, performance data, audio data, and the like. Also, R
The AM 3 can also store computer programs, performance data, audio data, and the like supplied from the outside via the external storage device 13 or the communication interface 14. The CPU 5 performs calculations or controls for editing or processing audio data in accordance with a computer program stored in the RAM 3 or the ROM 4.

The timer 6 supplies time information to the CPU 5. The CPU 5 can perform an interrupt process at predetermined time intervals according to the time information.

As described above, the MIDI interface 8 is connected to the MIDI interface 2 of the external tone generator 21.
3 is connected by a MIDI cable. The personal computer 1 can transmit performance data and audio data to the external sound source device 21 via the MIDI interface 8.

The communication interface 14 is connected to a communication network 41 such as a local area network (LAN), the Internet, and a telephone line. A server computer 42 is connected to the communication network 41. The personal computer 1 can receive voice data or a computer program from the server computer 42 via the communication network 41.

The external storage device 13 includes an interface for the external storage device, and is connected to the bus 2 via the interface. The external storage device 13 includes, for example, a floppy disk drive (FDD), a hard disk drive (HDD), a magneto-optical disk (MO) drive,
D-ROM (compact disk-read only memory) drive, digital versatile disk (DVD) device, and the like. Performance data and audio data are stored in the external storage device 1
3 or the RAM 3.

The computer program or the like may be stored in the external storage device 13 (for example, a hard disk) without being stored in the ROM 4. R from hard disk
By reading a computer program or the like into the AM 3, the CPU 5 can cause the CPU 5 to perform the same operation as when the computer program or the like is stored in the ROM 4. By doing so, the computer program or the like can be easily added or upgraded by copying the computer program or the like from another external storage medium such as a CD-ROM to the hard disk.

The communication interface 14 is connected to a communication network 41 such as a local area network (LAN), the Internet, or a telephone line, and is connected to a server computer 42 via the communication network 41. If no computer program or the like is stored in the external storage device 13, the server computer 4
2 can download a computer program or the like. The personal computer 1 serving as a client transmits a command for requesting download of a computer program or the like to the server computer 42 via the communication interface 14 and the communication network 41. The server computer 42 receives the command, distributes the requested computer program and the like to the personal computer 1 via the communication network 41, and the personal computer 1 receives the computer program and the like via the communication interface 14. Thus, the data is stored in the external storage device 13 to complete the download.

The personal computer 1 reproduces the performance data and audio data stored in the RAM 3 according to a predetermined tempo, and transmits the reproduced performance data and audio data to the external tone generator 2 via the MIDI interface 8.
Send to 1. The external sound source device 21 transmits the transmitted performance data and audio data to the MIDI interface 2.
3 and supplies the received performance data to the musical tone waveform synthesis circuit 29 to form an accompaniment musical tone signal, and supplies the received voice data to the voice synthesis circuit 32 to form a singing signal. That is, the personal computer 1 and the external sound source device 21 constitute a system for generating a singing accompanied by accompaniment music.

Next, the processing performed by the audio data processing device (personal computer 1) will be described.

FIG. 2 is a diagram showing the import process.
In the import processing, text data (lyrics data) 52 is assigned to the performance data 51, and the note plus data 5
3 is a process of generating the C.3. In this specification, the performance data 5
1, all of the text data 52 and the note plus data 53 are called audio data.

The performance data 51 has, for example, note events N1 to N8 corresponding to eight notes (notes). Each of the note events N1 to N8 includes, for example, note-on timing data, note number (pitch), velocity (volume), and gate time (time from note-on to note-off).

The text data 52 is, for example, “red (red) _i / sunset [yu! Hi] _ is ￥”. here,
() Indicates reading kana, [] indicates ruby, and “_,
! "And the like indicate lyrics separators, and" /, ￥ "and the like indicate breath symbols.

The exhalation symbol is a symbol indicating breathing. When there is an exhalation symbol, the preceding lyrics data is pronounced only for the gate time, and thereafter, it is in a silent state. When there is no exhalation symbol, a certain lyric data and the next lyric data are pronounced so as to be smoothly connected. That is, the game time is ignored.

The lyrics separator is lyrics data (first lyrics event) corresponding to one note (note event).
And the lyrics data (second lyrics event) corresponding to the next note. The lyric data 52 is decomposed into lyric events according to the lyric delimiters, and is assigned to each of the note events N1 to N8. For example, “N1, red”, “N2, ▽”, “N3, Yu”, “N4,
"N" and "N5 is ▽". Here, “▽” is exhalation data corresponding to the exhalation symbol “/, ￥”, indicates breathing, and also plays a role of lyrics separation as can be seen from the example of “N2, ▽”. A set of the note event and a lyric (character string) event or a phoneme string event (a phoneme string event will be described later) is called a note plus event (an event in which a lyric event is added to a note). Note Plus Data 5
3 is a group of note plus events.

In the note plus data 53, in consideration of the reading kana and ruby, the kanji is deleted and only the corresponding kana is left. When there is a reading kana symbol "()", the kanji "red" and the reading kana symbol "()" before () are deleted, and the kana "kana" in () is left. The same applies to the case of the ruby symbol “[]”.

The kana in the note plus data 53 is converted into a phoneme string. For example, the pseudonym “ka” is replaced by the phoneme string “CL
(7.5 ms) + kha (4 × 7.5 ms) + aj (infinite length) ”. Syllables such as "ka" can be decomposed into phoneme strings on the time axis. The phoneme is, for example, "C
L "," kha ", and" aj ".

The personal computer 1 stores a table for converting between a pseudonym and a phoneme sequence.
Using this table, conversion from kana to phoneme strings can be performed. However, conversion from kanji, symbols, alphanumeric characters, and the like to phoneme strings cannot be performed. This is because kanji, symbols, alphanumeric characters, and the like may have multiple ways of reading.

The kana is converted into a phoneme string, and kanji and symbols are left as they are. The personal computer 1 stores a dictionary for converting kanji and the like to kana. Using this dictionary, kanji and the like can be converted into kana, and the kana can be converted into phoneme strings. The personal computer 1 supplies a phoneme string or the like to the external sound source device 21 (FIG. 1). The external sound source device 21 receives a phoneme string or the like and generates a sound.

The personal computer 1 can use text data as lyrics data. For example, existing text data such as text data for karaoke can be used.

The lyrics delimiter is "_ ,!" and the like, and the exhalation symbol is "/, $" and the like. For example, when karaoke text data is used, both a line feed mark and a sub line feed mark can be set as lyrics delimiters, and a page break mark can be set as a breath mark. Since a plurality of types of lyrics delimiters can be set, both a line feed mark and a sub-line feed mark can be used as lyrics delimiters.

The line break mark is a mark indicating a line break of the lyrics when the lyrics are displayed on the screen. The sub-line feed mark is a mark for line feed used only when displaying on a small screen. That is, a line break is performed only with a line break mark on a normal screen, and a line break is performed with both a line break mark and a sub-line break mark on a small screen. For example, “_” can be used as a line feed mark, and “!” Can be used as a sub line feed mark. The lyrics delimiter and exhalation symbol are not limited to the case where a symbol inserted for use for another purpose, such as a line feed mark or a line break mark, is originally used as the lyrics delimiter or exhalation symbol. The symbol set and inserted arbitrarily may be used as a lyrics separator or a breath symbol.

FIG. 3 shows the CP of the personal computer 1.
9 is a flowchart illustrating a text import process performed by U. As described above (FIG. 2), the text import process is a process of assigning the text data 52 to the performance data 51 and generating the note plus data 53.

At step SA1, a text file to be imported is selected. The text file includes text data 52 (FIG. 2) and is stored in the external storage device 13 (FIG. 1).
And so on. The operator uses the mouse 10 or the like to
One or a plurality of text files can be selected.

At step SA2, a pointer is set at the position of the first note event N1 in the performance data 51. The performance data 51 is stored in the RAM 4 or the external storage device 13.
(FIG. 1).

At Step SA3, the text data 52
Set the pointer to the position of the first character code inside.
For example, the pointer is set at the position of “red” in the text data 52.

At step SA4, it is checked whether or not the character code matches one of the character codes set as the breath symbol. For example, “/, ￥” is set as the breath symbol. If they do not match, the flow advances to step SA5 according to the NO arrow.

At step SA5, it is checked whether or not the character code matches one of the character codes set as the lyrics delimiter. For example, “_ ,!” is set as the lyrics separator. If they do not match, the flow advances to step SA6 according to the NO arrow.

At step SA6, the character code is added to the contents of the character string register. String register is 1
This is a register for storing one or more character codes (hereinafter, referred to as lyric events) corresponding to notes, and initially stores nothing.

At step SA7, a pointer is set at the position of the next character code. For example, text data 5
The pointer is set at the position of “(” in 2. Then,
Returning to step SA4, the above processing is repeated.

If the character code at the position of the pointer set in step SA7 is "_" or "!", The flow advances to step SA5 via step SA4, where the character code "_" or "!" It is determined that it is a lyric delimiter, and the flow advances to step SA9 according to the YES arrow.

On the other hand, if the character code at the position of the pointer set in step SA7 is "/" or "$", it is determined in step SA4 that the character code "/" or "$" is a breath symbol. The process proceeds to Step SA8 according to the arrow of YES. In step SA8, an exhalation mark "@" is added to the contents of the character string register. afterwards,
Proceed to step SA9.

At step SA9, a reading pseudonym and ruby processing are performed. Specifically, when the lyric event stored in the character string register includes a reading kana or ruby, kanji or the like is deleted to leave a kana. Details of this processing will be described later with reference to the flowchart of FIG. Thereafter, the flow advances to Step SA10.

At step SA10, the contents of the character string register are added to the note event where the pointer is located,
Generate a note plus event. For example, the lyric event “Aka” in the character string register is added to the note event N1 to generate the note plus event “N1, Aka”.

At step SA11, the text data 5
Check if there is a continuation of 2. That is, it is checked whether the text data 52 ends. If not, follow the arrow YES and follow step SA1.
Proceed to 2.

At step SA12, the contents of the character string register are cleared to prepare for storing the next lyrics event.

At step SA13, a pointer is set at the position of the next note event in the performance data 51.
For example, a pointer is set at the position of the note event N2.

At Step SA14, the text data 5
The pointer is set to the position of the next character code in 2.
For example, a pointer is set at the position of the character code "i". Thereafter, the flow returns to step SA4, and the above processing is repeated.

When the processing of the last character code of the text data 52 is completed, the text import processing is completed according to a NO arrow in step SA11. By this processing, the note plus data 53 is generated.

In the above-described flowchart, the case where the text data 52 in the text file is imported has been described, but text included in a predetermined track in the standard MIDI file may be imported.

FIG. 4 is a flowchart showing details of the reading kana and ruby processing shown in step SA9 of FIG.

At step SB1, it is checked whether or not there is a character code of a kana surrounded by () or [] in the character string register. For example, “(red)” or “[Yuhi]” in the text data 52 corresponds to this.

In the case of "sunset [Yu! Hi] _",
Both the character string "sunset (Yu)" and the character string "hi" correspond to this. That is, if there is a kana character enclosed in single parentheses of () or [], this corresponds to this.

When the character code is present, the flow advances to step SB2 according to the YES arrow. If there is no such character code, the reading kana and ruby processing are terminated according to the arrow of NO.

At step SB2, the character code of the non-kana character before () or [] in the character string register is deleted. Non-kana character codes include kanji, symbols, and alphanumeric characters. For example, “red” or “sunset” in the text data 52
Remove.

At step SB3, () or [] in the character string register is deleted. As a result, for example, the character string “sunset [Yu] is converted to“ Yu ”, and the character string“ hi ”
Is converted to "hi". Thereafter, the reading kana and ruby processing ends, and the process returns to the text import processing of FIG.

By the above processing, the kanji and the like of the kanji and the like to which the reading kana and the ruby are assigned are deleted and the kana remains. Kanji and the like are character codes that are unnecessary for pronunciation.

FIG. 5 shows the CP of the personal computer 1.
It is a flowchart which shows the phoneme string conversion process which U performs.
The phoneme string conversion process is a process of converting the lyric event into a phoneme string after generating the note plus data 53 (FIG. 2).

In the display circuit 7, one note plus event is displayed on one line. That is, one row corresponds to one note plus event. In step SC1, a line to be converted is selected from the note plus data 53. The operator can select one line or a plurality of lines (that is, one or a plurality of note plus events) by using the mouse 10 or the like with reference to the display contents of each line displayed on the display circuit 7.

At step SC2, a pointer is set at the position of the first line in the selected line. For example, the pointer is set at the position of the note plus event “N1, red”.

At step SC3, it is checked whether or not the character string (lyric event) of the line is a kana only. If there is only a kana, the process proceeds to step SC4 according to a YES arrow to perform phoneme sequence conversion. If not only kana, N
According to the arrow O, without performing the phoneme sequence conversion, step SC6
Proceed to. If a breath symbol is included in addition to the pseudonym, it is determined that only the pseudonym is included.

In step SC4, a phoneme string corresponding to the character string is determined with reference to a kana-phoneme string table.
For example, a phoneme string corresponding to the character string "red" is obtained.

At step SC5, a character string is replaced with a phoneme string. That is, in the note plus data 53, the character string "red" is deleted, and the corresponding phoneme string is written. Thereafter, the process proceeds to Step SC6.

At Step SC6, it is checked whether or not the next line exists in the selected range line. If there is only one selection range row, there is no next row, so the phoneme string conversion processing is terminated according to the arrow of NO. YES if there is a next line
Then, the flow advances to Step SC7.

At step SC7, a pointer is set at the position of the next line. For example, note plus event "N
The pointer is set at the position of "2, ii". Thereafter, the flow returns to step SC3, and the above processing is repeated. The breath symbol “▽” is not converted into a phoneme sequence. Only the pseudonym "i" is converted to a phoneme sequence.

When the processing for all the rows is completed, step S
In step C6, the phoneme sequence conversion process is ended according to the arrow of NO. In a later process, the converted phoneme sequence is supplied to the external sound source device 21 (FIG. 1) and is emitted.

After being converted into a phoneme string, the phoneme string is converted again into a character string, and both the phoneme string and the character string are displayed on a screen on the display circuit 7. The operator can know the character string and the corresponding phoneme string. It is also possible to know whether or not the phoneme string conversion has been performed normally.

Next, the blank line insertion processing will be described. As described above, after assigning a lyric event to a note event, there may be a case where it is desired to insert a new lyric between the lyrics. For example, a part of the lyrics is missing, and the note event and the lyrics event are associated with each other in a state where the lyrics after the missing part are packed forward. As a result, the correspondence between the note event and the lyrics event is incorrect. And a note plus event has been generated. In that case, a blank line may be inserted at a desired location, and new lyrics may be embedded in the blank line.

FIGS. 6A to 6D are diagrams for explaining blank line insertion processing. FIG. 6A shows an example of note plus data. The note plus data has, for example, eight lines of note plus events. The note plus event on the first line is a note event N1 and a lyrics event L
One.

The operator can use a mouse or the like to specify the position PT where the blank line is to be inserted and the number of lines LL.
For example, the third line can be specified as the insertion position PT, and two lines can be specified as the number LL of insertion lines.

First, the lyrics events L3 to L8 after the insertion position PT are copied to the buffer shown in FIG. 6B. Note that a blank line may be included in the buffer.

Next, as shown in FIG.
The lyrics event is erased from the insertion position PT (third line) of the note plus data of (A) by the number of inserted lines LL (two lines). That is, the lyrics events L3 and L4 are deleted.

Next, as shown in FIG.
The lyrics event in the buffer (FIG. 6B) is copied to the fifth and subsequent lines in the note plus data of (C). However, since there is no note event after the note event N8, the remaining lyrics events are collectively assigned to the note event N8. That is, the lyric events L6 to L8 are assigned to the note event N8.

In the note plus data of FIG. 6D, a lyric event in a blank line is assigned to the note events N3 and N4. As a result, two blank lines have been inserted. Then, the note plus event after the blank line insertion point is in a state where the association between the note event and the lyrics event has been changed. For blank lines,
Since the lyric event has been deleted, it becomes a normal note event instead of a note plus event.

FIG. 7 is a flowchart for realizing the above-described blank line insertion processing. In step SD1, a blank line insertion position PT and an insertion line LL are specified.

At step SD2, the lyric event (phoneme string / character string) after the insertion position PT is buffered (FIG. 6).
(B)).

At step SD3, the lyric event corresponding to the number LL of inserted lines is deleted from the insertion position PT (FIG. 6).
(C)).

In step SD4, a buffer (FIG. 6)
The lyric event of (B)) is assigned to a note event after the insertion line number LL from the insertion position PT. At this time, if the lyrics event remains in the buffer even though there are no more note events to be assigned, all the remaining lyrics events are added to the last note event. This completes the blank line insertion process.

The method of generating the note-plus data of FIG. 6D by inserting a blank line into the note-plus data of FIG. 6A may be a method other than the above.

FIGS. 8A to 8F are views for explaining the lyrics event insertion processing. FIG. 8A shows an example of note plus data. The operator can specify the range RG of the lyric event to be inserted using a mouse or the like. For example, as the range RG, the lyrics event L5
And L6 can be specified.

First, the lyric events L5 and L6 in the range RG are copied to the clipboard shown in FIG. 8B.

Next, as shown in FIG. 8C, the operator can designate the insertion position PT using a mouse or the like. For example, the note plus event “N3, L3” on the third line can be specified as the insertion position PT.

Next, the lyrics events L3 to L8 after the insertion position PT are copied to the buffer shown in FIG. 8D.

Next, as shown in FIG.
(C) Note events N3 and N after the insertion position PT
4, the lyrics event L on the clipboard (FIG. 8B)
5 and L6 are assigned.

Next, as shown in FIG.
The fifth line “N5, L” in the note plus data of (E)
After "5", lyrics events L3 to L8 in the buffer (FIG. 8D) are assigned. However, note event N8
Since there is no note event after that, the remaining lyrics events L6 to L8 are collectively assigned to the note event N8. As a result, the lyrics events L5 and L6 are inserted at the insertion position PT in the note plus data in FIG.

FIG. 9 is a flowchart for realizing the above-described lyric event insertion processing.

In step SE1, a lyric event having an arbitrary number of lines RG is designated (FIG. 8A) and copied to the clipboard (FIG. 8B).

In step SE2, the insertion position PT of the lyric event is designated (FIG. 8C).

At step SE3, the lyric event after the insertion position PT is copied to the buffer (FIG. 8D).

At step SE4, the lyric event on the clipboard (FIG. 8B) is allocated to the note plus data by the number of insertion lines RG from the insertion position PT (FIG. 8).
(E)).

At step SE5, the buffer (FIG. 8)
The lyrics event of (D)) is assigned to the note plus data after the insertion line number RG from the insertion position PT (FIG. 8 (F)). Thus, the lyric event insertion processing ends.

Note that a method other than the above may be used to insert the desired lyric event into the note plus data shown in FIG. 8A and generate the note plus data shown in FIG. 8F.

FIGS. 10A to 10E are diagrams for explaining the lyrics event deletion processing. FIG. 10A shows an example of note plus data. The operator can specify the range RG of the lyrics event to be deleted using a mouse or the like. For example, as a range RG, a lyrics event L
3 and L4 can be specified.

First, the clipboard shown in FIG.
The lyrics events L3 and L4 within the range RG are copied.
The contents of this clipboard are not used in the subsequent processing, but can be used as the clipboard in FIG. 8B as needed.

Next, in the buffer shown in FIG.
The lyrics events L5 to L8 behind G are copied.

Next, as shown in FIG.
The lyrics events L3 to L8 after the range RG in (A) are deleted.

Next, as shown in FIG.
In note events N3 to N8 after the range RG in (D),
Lyrics events L5 to L8 in the buffer (FIG. 10C)
Assign. However, since the lyrics events L3 and L4 have been deleted, the final note events N7 and N8 are assigned the lyrics events in blank lines. In the note plus data of FIG. 10E, the lyric event L3 follows the lyric event L2 because the lyric event L3 and N4 have been deleted.

FIG. 11 is a flowchart for realizing the above-mentioned lyric event deletion processing.

In step SF1, a lyric event having an arbitrary number of lines RG to be deleted is designated (FIG. 10A) and copied to the clipboard (FIG. 10B).

In step SF2, the lyric event after the line RG to be deleted is copied to the buffer (FIG. 10C).

In step SF3, the lyrics data after the line RG to be deleted is deleted (FIG. 10D).

In step SF4, a buffer (FIG. 10)
The lyric event of (C)) is assigned to the note plus data after the deleted line RG (FIG. 10E). Thus, the lyric event deletion processing ends.

The lyric event deletion process is effective when the performance data is arranged and the number of note events decreases, or when the number of lyric events increases due to erroneous input of the lyric data.

The method for generating the note-plus data shown in FIG. 10E by deleting a desired lyric event from the note-plus data shown in FIG. 10A may be a method other than the above.

FIGS. 12A to 12D are diagrams for explaining the first lyrics automatic assignment processing.

FIG. 12A shows an example of note plus data. The operator can use a mouse or the like to specify the range RG of the note plus event to be automatically assigned. For example, the second to sixth note plus events can be designated as the range RG.

First, in the buffer shown in FIG.
Copy the lyrics event “Iue Okakiku” in G.

Next, as shown in FIG.
The lyrics event in the range RG of (A) is deleted.

Next, as shown in FIG.
The lyric event “Iue Okakiku” in the buffer (FIG. 12B) is assigned one character at a time to the note events N2 to N6 in the range RG of (D). However, the remaining character string “Kakiku” is assigned to the last note event N6. In the note plus data of FIG.
One character is assigned to each of the note events N2 to N5, and three characters “Kakiku” are assigned to the note event N6. Basically, one character (syllable) is assigned to one note.

The first lyrics automatic assignment processing is performed in the range RG
Is a process of automatically assigning a lyric event to each line without increasing or decreasing the number of lines.

FIG. 13 is a flowchart for realizing the above-mentioned first automatic lyrics assigning process.

At step SG1, a plurality of lines RG to which lyrics are automatically assigned are selected (FIG. 12A).

At step SG2, the selected row R
Buffer the lyrics event contained in G (FIG. 12 (B))
Copy to However, when the lyrics event is composed of phoneme strings, the phoneme strings in the buffer are converted into character data.

At step SG3, the selected plural rows R
The lyric event in G is deleted (FIG. 12C).

In step SG4, the buffer (FIG. 12)
The character string data in (B)) is assigned to the note-plus data of the selected plurality of rows RG (FIG. 12D). This is the end of the first lyrics automatic assignment processing.

FIGS. 14A and 14B are diagrams for explaining blank line automatic insertion processing. FIG. 14A shows an example of note plus data. The operator can select a row RG into which a blank row is to be inserted using a mouse or the like. For example, a note plus event in the fourth row can be specified as the selected row RG.

First, the number of characters constituting the lyrics event in the selected row RG is counted. For example, the number of characters in the lyric event “Okakiku” is four.

Next, as shown in FIG. 14B, a blank line of "the number of characters-1" is inserted at the position of the selected line RG. For example, 4-1 = 3 blank lines are inserted. A blank line is assigned to the note events N4 to N6 after the selected line RG, and a lyric event “Okakiku” is assigned to the note event N7.

Note that the present invention is not limited to the case where a blank line is inserted before the lyric event “Okakiku” of the selected line RG, and a blank line may be inserted after the lyric event “Okakiku”.

Thereafter, the fourth to seventh rows are selected, and FIG.
By performing the first lyrics automatic assignment process shown in FIG. 2, a lyrics event can be assigned to the blank line. Specifically, "N4,""N5,""N6,"
It can be assigned as “N7, ku”.

FIG. 15 is a flowchart for realizing the above-described blank line automatic insertion processing.

In step SH1, a row RG for automatically inserting a blank row is selected (FIG. 14A).

At step SH2, the number of characters of the lyric event in the selected row RG is detected. When the lyric event is composed of a phoneme sequence, the phoneme sequence is converted into characters and then the number of characters is detected.

At step SH3, blank lines of the number of "number of characters-1" are inserted at the position of the selected line RG. This insertion can be performed using a buffer, similarly to the blank line insertion. This completes the blank line automatic insertion process.

FIGS. 16A to 16D are views for explaining the second lyrics automatic assignment processing.

FIG. 16A shows an example of note plus data. The operator can use a mouse or the like to specify the range RG of the note plus event to be automatically assigned. For example, the second to sixth note plus events can be designated as the range RG.

First, in the buffer shown in FIG.
Copy the lyrics event “Iue Okakiku” in G.

Next, as shown in FIG. 16C, the number of characters (7 characters) in the buffer is two characters larger than the number of lines (5 lines) in the range RG, so that two blank lines are added to the range RG. Insert in position.

Next, as shown in FIG. 16D, the number of note events N2 to N8 from the position of the selected row RG to the number of characters (seven characters) in the buffer (FIG. 16B) is added to the buffer event. Is assigned one character at a time. A lyric event of one character is assigned to each of the note events N2 to N8.

The second lyrics automatic assignment process is a process for ensuring that all characters included in the selected line RG are assigned one character per one note event.

FIG. 17 is a flowchart for realizing the second lyrics automatic allocation process.

In step SI1, a note plus event of a plurality of lines RG to which lyrics are automatically assigned is selected (FIG. 16A).

In step SI2, the selected plural rows R
The lyric event in G is copied to the buffer (FIG. 16B). However, when the lyrics event is composed of phoneme strings, the phoneme strings in the buffer are converted into character data.

At Step SI3, the number of characters in the buffer is detected. In step SI4, the selected row number RG
And the number of characters detected.

At step SI5, it is checked whether the number of compared rows is equal, large or small. If the number of lines is large, the process proceeds to step SI6, where the number of extra lines is deleted, and the process proceeds to step SI8. Step S when the number of rows is small
Proceed to I7 and insert the number of missing lines (FIG. 16C),
Proceed to step SI8. When the number of rows is equal, the process proceeds to step SI8 without changing the number of rows.

In step SI8, the character string data in the buffer is transferred from the head row of the selected row RG by the number of selected rows RG, by the number of deleted rows, or by the number of inserted rows.
It is assigned to the note plus event (FIG. 16D).
With the above, the second lyrics automatic assignment processing ends.

FIGS. 18A to 18D are diagrams for explaining a plurality of rows merging process. FIG. 18A shows an example of note plus data. The note plus data includes the note plus events "N1, A", "N2, I", "N3,"
"," N4, eh ", and" N5, oh ".

The operator can specify the range RG of the note plus event to be merged using a mouse or the like. For example, the second and third note plus events can be designated as the range RG.

First, the range R is stored in the buffer shown in FIG.
Copy the lyric event “say” in G.

Next, as shown in FIG.
Only the lyric event “I” on the first line in G is left, and the lyric event “U” on the remaining lines is deleted. The lyric event "E, O" after the range RG is packed into the previous line. Note plus data is for note plus event "N1,
"A", "N2, I", "N3, E", "N4, O",
“N5, _”.

Next, as shown in FIG.
The character string “say” in the buffer is assigned to the note event N2 of the first line in G. Second note event N2
, A lyric event “say” in the range RG is merged and assigned.

In the above-described first or second automatic lyric allocation processing, a single-character lyric event is allocated to one note event. Thereafter, by using this multiple-line merging processing, it can be assigned to one note event. It can be modified to assign lyric events of two or more characters.

FIG. 19 is a flowchart for realizing the above-described multiple-row merging process. In step SJ1,
Select multiple lines RG to merge lyrics events (FIG. 18)
(A)).

In step SJ2, the selected plural rows R
The lyric event in G is copied to the buffer (FIG. 18B). However, when the lyrics event is composed of phoneme strings, the phoneme strings in the buffer are converted into character data.

At step SJ3, the selected plural rows R
Of G, the lyric event of the remaining line is deleted except one line (FIG. 18C). Lyrics events after deletion are padded to the previous line.

In step SJ4, the character string data in the buffer is assigned to the note event in the note plus data of the remaining line in the range RG (FIG. 18 (D)). This is the end of the multiple-row merging process.

FIGS. 20A and 20B are diagrams for explaining the lyrics event division processing. FIG. 20A shows an example of note plus data. Note Plus data is
Note plus events "N1, A", "N2, say",
"N3, h".

The operator uses a mouse or the like to select a line RG where the lyric event is to be divided, and sets the cursor CS at a character position within the line where the lyric event is to be divided. For example,
As the range RG, the second note plus event “N2,
Is selected, and the cursor CS is set between "i" and "u". After that, the operator presses the return key (execute key).
Is operated, the following division processing is performed.

As shown in FIG. 20B, the characters "i" and "u" are divided. The note plus data includes the note plus events "N1, A", "N2, I", "N3,"
"". The character "U" after the cursor CS is
This is merged with the lyric event “e” in the line after the selected line RG.

It is convenient to insert a large number of lyric events in blank lines (FIG. 6) and assign lyrics in order from the beginning.

FIG. 21 is a flowchart for realizing the above-mentioned lyrics event division processing.

At step SK1, a row RG containing the lyrics event to be divided is selected (FIG. 20A).

At step SK2, the cursor CS is set at the division position by mouse operation or the like (FIG. 20A).

In step SK3, in response to the operation of the return key, the lyric event after the division position is added to the lyric event in the note plus data of the next line (FIG. 2).
0 (B)). Thus, the lyric event division processing ends.

FIGS. 22A to 22C are diagrams for explaining the kana conversion process. FIG. 22A shows an example of note plus data. The note plus data includes the note plus events “N1, Aka”, “N2, I”, “N3,
Evening "and" N4, Sun ".

The operator uses the mouse to move the mouse pointer 71 to the position of the lyric event “evening” where the kana conversion is to be performed.
Move and click the right mouse button.

As shown in FIG. 22B, the reading kana candidate of the lyric event “evening” is displayed on the menu 72. In the menu 72, for example, "Yu" and "Cough" are displayed. The operator moves the mouse pointer 71 to the position of the character string “Yu”, which is an appropriate reading kana, and clicks the left mouse button.

As shown in FIG. 22C, the lyric event assigned to the note event N3 is converted from the kanji character "Yu" to the pseudonym "Yu". In addition to kanji, alphanumeric characters and symbols can be converted to kana. If kanji or alphanumeric characters are converted to kana, then kana can be converted to phoneme strings (FIG. 5), so that phoneme strings can be pronounced.

FIG. 23 is a flowchart for realizing the above-described kana conversion processing. In step SL1, a line of a character to be converted to a kana is designated by the mouse pointer 71 (FIG. 22A).

At step SL2, when the right button of the mouse is clicked, the reading kana corresponding to the selected character is searched from the database and displayed as a reading kana candidate (FIG. 22B). The database is a dictionary for converting kanji and the like to kana, and is stored in the external storage device 13 or the RAM.
3 and so on.

At step SL3, one of the reading kana candidates is designated by the mouse pointer 71 (FIG. 22).
(B)).

In step SL4, when the left button of the mouse is clicked, the designated kana candidate is assigned to the note event in the note plus data as the character string of the designated line (FIG. 22 (C)). Thus, the kana conversion process ends.

By supplying the note-plus event created as described above (a character string converted to a phoneme string by the phoneme string conversion process of FIG. 5) to the external sound source device 21, Voice synthesis circuit 32
, A singing voice signal can be generated. Note that the audio data processing device (personal computer 1) described in the present embodiment is a dedicated audio data processing device that can reproduce and process note-plus data in the same format. This audio data processing device uses note events and lyric events (phoneme strings)
Can be sent in an appropriate order.

Here, an appropriate sequence for transmitting the note event and the lyrics event will be described. FIG.
(A), (B) is a diagram for explaining the transmission order of the note event and the lyrics event. The note plus event is, for example, a pair of a note event N1 and a lyric event L1 (FIG. 6A). Although the note event N1 and the lyric event L1 should be transmitted at the same timing in theory, since MIDI communication is serial communication, it is important to determine which should be transmitted to the external tone generator 21 (FIG. 1) first. . The note event N1 is converted into a note-on event and transmitted. The lyric event is converted into a phoneme string (FIG. 5) and transmitted.

FIG. 24A shows the timing of transmission in the order of phoneme strings and note-on. First, a phoneme string “aj” corresponding to the syllable “a” is transmitted. Thereafter, when a note-on is transmitted, the sound of "A" starts at that timing. Next, the phoneme string “ij” corresponding to the syllable “i” is transmitted. Thereafter, when a note-on is transmitted, the sound is switched from “A” to “I” at that timing. Next, the phoneme string “uj” corresponding to the syllable “u” is transmitted. Thereafter, when a note-on is transmitted, the sound is switched from "i" to "u" at that timing.

In FIG. 24A, note-on (note event) and phoneme string (lyric event) are associated with each other, so that an appropriate song can be sung.

FIG. 24B shows the timing for transmitting in the order of note-on and phoneme sequence. First, a note-on is sent. At this point, since the phoneme string has not been set, the phoneme string which is initially set, for example, "A" starts to sound. After that, the phoneme sequence “a” corresponding to the syllable “a”
j ". At this time, there is no change in pronunciation, and the initially set pronunciation of "A" is maintained. Next, when a note-on is transmitted, the pronunciation of the phoneme string “A” initially set at that timing is switched to the pronunciation of the transmitted phoneme string “A”. Thereafter, the phoneme string “ij” corresponding to the syllable “i” is transmitted, but the pronunciation is not affected. Next, when a note-on is transmitted, the sound is switched from “A” to “I” at that timing. After that, the phoneme string “uj” corresponding to the syllable “u” is transmitted, but the pronunciation is not affected.

In FIG. 24B, there is no correspondence between note-on (note event) and phoneme string (lyric event), and the lyrics are produced with a delay of one note. This makes it impossible to sing a proper song.

As shown in FIG. 24A, it is necessary to transmit a phoneme sequence to the external tone generator 21 first, and then transmit a corresponding note-on. As described above, the audio data processing device according to the present embodiment is a device that can correctly handle note-plus data, and transmits a phoneme sequence before transmitting a note-on for each note-plus event. It is supposed to. However, a sequencer or the like other than the audio data processing device cannot correctly handle the note plus event. For this reason, the audio data processing apparatus has a function of converting the note plus data into a more general-purpose standard MIDI file. At this time, simply converting to a standard MIDI file may cause inconvenience as described later. Therefore, the conversion is performed in consideration of the transmission order as described above. Hereinafter, a method of converting the note plus data into the standard MIDI file in consideration of the transmission order of the note event and the lyrics event (phoneme sequence) will be described.

FIG. 25 is a diagram for explaining a method of converting the note plus data 75 into a standard MIDI file 76.

The note plus data 75 includes a note event N1, a lyric event L1, a duration T2,
It has a note event N2, a lyrics event L2, and a duration T3.

The note event N1 has a note-on event NON1 and a gate time GT1. The note-on event NON1 includes, for example, a note number (pitch) and a velocity (volume). The gate time GT1 is a time from note-on to note-off, for example, 450
It is. The lyrics event L1 is composed of a phoneme sequence. However, a breath symbol, a Chinese character, and the like are included in the lyrics event L1 as a character string.

The note event N2 is the note event N
As in the case of No. 1, it has a note-on event NON2 and a gate time GT2. The gate time GT2 is, for example, 22
0. The lyric event L2 is composed of a phoneme sequence, like the lyric event L1.

The duration T2 is from the start of note-on event NON1 to the note-on event NON2.
, For example, 480. The duration T3 is a time from the start of sounding of the note-on event NON2 to the start of sounding of the next note-on event, and is, for example, 240. The values of the gate time and the duration are represented by the number of clocks. One clock is, for example, 1/480 of a quarter note length.

The standard MIDI file 76 has M
This is a general-purpose format file suitable for the IDI standard. The standard MIDI file 76 includes a pair of a duration and an event.

The standard MIDI file 76 includes, in order, the lyrics event L1, the duration TT1, the note-on event NON1, the duration TT2, the note-off event NOFF1, the duration TT3, the lyrics event L2, the duration TT4, the note-on event NON2, the duration TT5, and the note. It has an off event NOFF2 and a duration TT6.

The lyrics event L1 and the note-on event NON1 are the same as the lyrics event L1 and the note-on event NON1 in the note plus data 75.
The order is different. By transmitting the lyric event L1 prior to the note-on event NON1, FIG.
A normal song can be sung as shown in FIG.

The duration TT1 is a lyrics event L
This is the time from the transmission of 1 to the transmission of the note-on event NON1, for example, 5. If the lyrics event L1 is located at an address before the note-on event NON1, the duration TT1 can theoretically be considered to be 0.

However, when the duration TT1 is set to 0, the transmission order may differ depending on the personal computer 1 or the sequencer transmitting the standard MIDI file. That is, the duration TT1 is set to 0
Then, lyrics event L1, note-on event NO
The message is not always transmitted in the order of N1, but may be transmitted in the order of the note-on event NON1 and the lyrics event L1. Further, a similar phenomenon occurs when the duration TT1 is not 0 but 1 or 2.

[0210] This is because the sequencer or the like has a lyric event L1.
The importance of the note-on event NON1 is compared with that of the note-on event NON1, and it is determined that the importance of the note-on event NON1 is high, and the note-on event NON1 is transmitted before the lyrics event L1. The lyric event can be transmitted by a system exclusive message determined by the MIDI standard.

In order to prevent the above-mentioned problems, the duration TT1 is set to 5. Although the duration TT1 is preferably 3 or more, an excessively large value may adversely affect the previous event. Duration TT1 is 3
To 10 are preferred. However, this is the case where the note resolution of the clock is 1/480 of the quarter note length, and when the note resolution is different, the above preferred numerical value takes another value. For example, if the note resolution of the clock is 1 / 96th of a quarter note, about 1-3 is preferable.

The duration TT2 is a note-on NO
This is a time from N1 to note-off NOFF1, and corresponds to the gate time GT1 of the note plus data 75, for example, 450.

The duration T2 in the note plus data 75 is from the note on NON1 to the next note on NO.
This is the time until N2, which is decomposed into durations TT2, TT3 and TT4 in the standard MIDI file 76.

The duration TT2 is, as described above,
It is 450 as in the gate time GT1 of the note plus data 75. The duration TT4 is the time from the transmission of the lyrics event L2 to the transmission of the note-on event NON2, and is 5 like the duration TT1. Duration TT3 is a note-off event NO
This is the time from the transmission of FF1 to the transmission of the lyrics event L2, and is represented by TT3 = T2-TT2-TT4. That is, TT3 = 480−450−5 = 25.

As described above, the lyric event L1 is located at the address before the note-on event NON1,
By setting the duration TT1 to 5, the lyric event L1 can be reliably transmitted earlier than the note-on event NON1. At this time, the timing of the lyric event L1 is advanced by 5 without changing the timing of the note-on event NON1, so that the note-on NO
There is no deviation of the sounding timing by N1.

Further, by converting the note plus data 75 into a standard MIDI file 76, the versatility is increased, and processing can be performed by other sequencers, etc., and the standard MIDI file 76 is supplied to the user via a floppy disk or the like. It becomes possible to do.

As shown in FIG. 26A, the standard MIDI file (SMF) conversion means 81 can convert the note plus data 75 into SMF data. The personal computer 1 includes an SMF conversion unit 8
In step 1, note plus data 75 is standard M
It can be converted into an IDI file 76 and saved in the external storage device 13. Standard MID saved
The I file 76 can be used in another personal computer, sequencer, or the like. Further, when the personal computer 1 transmits the standard MIDI file 76 to the external tone generator 21, the external tone generator 21 performs sound generation processing according to the standard MIDI file 76.

Further, as shown in FIG. 26B, the note-plus-conversion means 82 can convert the SMF data 76 into the note-plus data 75. The personal computer 1 can load the standard MIDI file 76 stored in the external storage device 13, convert the standard MIDI file 76 into note-plus data 75 in the note-plus conversion means 82, and store the converted data in the RAM 3. At this time, on the contrary to the conversion of the note plus data 75 into the standard MIDI file 76, a process of combining the lyrics events stored independently of the note events into one note plus event is performed. First, the data in the standard MIDI file is searched sequentially from the top,
When a note-on event is found, a corresponding note-off event is searched for to generate one note-on event including a gate time. The lyric event existing up to 5 clocks before the note-on event is searched, and if found, the lyric event is added to the note-on event, and a note-plus event that combines the note-on event and the lyric event is added. Generate an event. The operator can perform various edits on the note plus data 75 as described above. Note that a process similar to the process for converting the note plus data 75 into a standard MIDI file may be applied to a case where the data is converted into a relatively versatile data format other than the standard MIDI file.

Note that the present invention is not limited to the case where the speech synthesis circuit in the external sound source device 21 produces a phoneme string. A sound source board including a voice synthesis circuit may be inserted into the personal computer 1, and the sound source board may emit a phoneme sequence. In that case, the personal computer 1 has the external tone generator 2
1 does not need to be connected.

The audio data processing device according to the present embodiment is not limited to the form using a personal computer and application software, but may be an electronic musical instrument or a sequencer. The application software may be stored in a storage medium such as a magnetic disk, an optical disk, or a semiconductor memory and supplied to a personal computer, or may be supplied via a network.

[0221] The format of the audio data is set to the occurrence time of a performance event such as a standard MIDI file by one.
The event is not limited to the “event + relative time” represented by the time from the previous event, but the “event + absolute time” that represents the occurrence time of the performance event in absolute time within the song or bar,
"Pitch (rest) + note length", which represents performance data in note pitch and note length or rest and rest length, secures a memory area for each minimum resolution of performance and generates performance events A format such as a “solid method” in which performance events are stored in a memory area corresponding to time may be used.

The audio data may be in a format in which data of a plurality of channels are mixed, or in a format in which data of each channel is separated for each track.

Although the present invention has been described in connection with the preferred embodiments,
The present invention is not limited to these. For example, it will be apparent to those skilled in the art that various modifications, improvements, combinations, and the like can be made.

[0224]

As described above, according to the present invention,
Since text data including kana or kanji can be processed, input or editing of text data is easy. Existing text data in which kana or kanji are mixed can be used.

Further, when text data is assigned to note data, both the line feed mark and the sub line feed mark can be used as delimiter data in the text data for karaoke.

[0226] Also, the audio data is, for example, standard M
Since it can be handled in the IDI file format, versatility is increased and MIDI communication can be performed.

[Brief description of the drawings]

FIG. 1 is a diagram showing a hardware configuration of a system in which a personal computer and an external sound source device are connected.

FIG. 2 is a diagram illustrating a process of assigning a text event to a note event.

FIG. 3 is a flowchart illustrating a text import process.

FIG. 4 is a flowchart showing details of the reading kana and ruby processing shown in step SA9 of FIG. 3;

FIG. 5 is a flowchart showing a phoneme string conversion process.

FIG. 6 is a diagram for explaining a blank line insertion process.

FIG. 7 is a flowchart showing a blank line insertion process.

FIG. 8 is a diagram illustrating a lyric event insertion process.

FIG. 9 is a flowchart showing lyrics event insertion processing.

FIG. 10 is a diagram illustrating a lyric event deletion process.

FIG. 11 is a flowchart showing lyric event deletion processing.

FIG. 12 is a diagram for describing a first lyrics automatic assignment process.

FIG. 13 is a flowchart showing a first automatic lyrics assigning process.

FIG. 14 is a diagram illustrating a blank line automatic insertion process.

FIG. 15 is a flowchart illustrating blank line automatic insertion processing.

FIG. 16 is a diagram illustrating a second automatic lyrics assigning process.

FIG. 17 is a flowchart showing a second lyrics automatic assignment process.

FIG. 18 is a diagram for explaining a multiple-line merging process;

FIG. 19 is a flowchart showing a multiple row merging process.

FIG. 20 is a diagram illustrating a lyric event division process.

FIG. 21 is a flowchart showing lyrics event division processing.

FIG. 22 is a diagram for explaining kana conversion processing.

FIG. 23 is a flowchart showing a kana conversion process.

FIG. 24 (A) shows a sound produced when transmitted in the order of a phoneme string and a note-on, and FIG. 24 (B) shows a sound produced when transmitted in the order of a note-on and a phoneme string. .

FIG. 25: Note Plus Data and Standard MID
It is a figure showing conversion to and from I file.

26A is a diagram illustrating a standard MIDI file conversion unit, and FIG. 26B is a diagram illustrating a note plus conversion unit.

[Explanation of symbols]

1 personal computer, 2 bus, 3
RAM, 4 ROM, 5 CPU, 6 timer, 7 display circuit, 8 MIDI interface, 9, 11 detection circuit, 10 mouse,
12 keyboard, 13 external storage device,
14 communication interface, 21 external sound source device,
22 bus, 23 MIDI interface, 24 RAM, 25 ROM, 26 CP
U, 28 display circuit, 29 musical tone waveform synthesis circuit,
31 sound system, 32 voice synthesis circuit,
33 detection circuit, 34 switch, 41
Communication network, 42 server computers,
51 performance data, 52 text data,
53 Note Plus Data, 75 Note Plus Data, 76 Standard MIDI File, 81
Standard MIDI file conversion means, 82
Note Plus conversion means

Claims (18)

[Claims] 1. A storage means for storing text data including a character string and note data including data in note units, wherein a plurality of types of characters are included in the text data as delimiter data indicating a delimitation of a character string. A sound data processing device comprising: a search unit that searches from included character strings and assigns a character string delimited by the searched character to data of one note in the note data to generate audio data. 2. A storage means for storing text data including a character string and note data including data in musical note units, wherein a plurality of types of characters are used as breath symbols in the character string included in the text data. And a generating means for generating a voice data by adding a breath mark to a character string delimited by the searched character and assigning it to data of one note in the note data. 3. A storage means for storing text data including data in which a non-kana character and its reading kana or ruby are paired, and note data including data in note units, discarding the non-kana characters in the text data. Generating means for generating audio data by assigning only the reading kana or ruby to note data in the note data. 4. An audio data processing device comprising: storage means for storing audio data in which text data and note data are paired; and changing means for changing the combination of the text data and note data for writing. . 5. A storage unit for storing text data including a character string composed of a plurality of characters and note data including data in note units, and converting one character in the text data into one note data in the note data. Generating means for generating voice data by allocating the voice data. 6. A first data in which a first character string data and a first note data are paired, and a second data in which a subsequent second character string data and a second note data are paired. Storage means for storing data of the first character string data; designation means for designating a division position in the first character string data; and dividing the first character string data into two at the division position, Processing means for merging the second character string data with the second character string data. 7. A storage means for storing voice data in which character data and note data are paired, and if the character data is a kanji, an alphanumeric character or a symbol, a kana candidate for pronouncing the character data is stored. An audio data processing apparatus comprising: a presenting means for presenting; and a changing means for designating one kana from the kana candidates, changing the character data to the kana, and storing the data in the storage. 8. A storage means for storing voice data in a format in which lyrics data and note data are paired, and wherein the lyrics data paired with the note data is located a minute before the note data. Recording means for converting the voice data and recording the lyrics data and the note data as voice data in an independent format. 9. A storage means for storing audio data in which lyrics data and note data are independent from each other, and wherein the lyrics data paired with the note data is located a minute before the note data. An audio data processing device comprising: a generation unit configured to read the audio data from a storage unit and generate audio data in a format in which the lyrics data and the note data are combined. 10. A recording medium of a computer program for processing text data including a character string and note data including data in musical note units, wherein a plurality of types of characters are used as delimiter data indicating a delimiter of a character string. To cause a computer to execute a procedure of searching a character string included in text data and assigning a character string delimited by the searched character to data of one note in the note data to generate voice data. A medium that records the program. 11. A recording medium for a computer program for processing text data including a character string and note data including note-by-note data, wherein a plurality of types of characters are included in the text data as breath symbols. A process for searching a character string, adding a breath mark to a character string delimited by the searched character, and assigning the character to data of one note in the note data to generate voice data, and causing the computer to execute the procedure. Medium on which program is recorded. 12. A recording medium of a computer program for processing text data including data in which a non-kana character and its reading kana or ruby are paired and note data including data in note units, wherein A medium in which a program for causing a computer to execute a procedure of discarding kana and assigning only the reading kana or ruby to note data in the note data to generate audio data is recorded. 13. A computer comprising: (a) a step of reading audio data in which text data and note data are paired; and (b) a step of writing by changing the combination of the text data and note data. A medium that stores a program to be executed. 14. A procedure for reading (a) text data including a character string composed of a plurality of characters and note data including data in note units; and (b) replacing one character in the text data with one in the note data. A medium for recording a program for causing a computer to execute a procedure of generating audio data by assigning to note data. 15. A first data in which first character string data and first note data are paired, and a second data in which subsequent second character string data and second note data are paired. (A) a step of specifying a division position in the first character string data; and (b) a step of dividing the first character string data by two in the division position. And a program for causing a computer to execute a procedure of dividing the second character string data into the second character string data. 16. A recording medium for a computer program for processing voice data in which character data and note data are combined, wherein (a) when the character data is a kanji, an alphanumeric character or a symbol, the character data is Causing the computer to execute a procedure of presenting a candidate kana for pronunciation, and (b) designating one kana from the kana candidates, changing the character data to the kana, and writing. Recording a program for the computer. (A) a procedure for reading out audio data in which lyric data and note data are paired; and (b) a lyric data paired with the note data is positioned a minute before the note data. A program for causing a computer to execute a procedure of converting the audio data and recording the lyrics data and the note data as audio data in an independent format. (A) reading out audio data in which the lyrics data and the note data are independent audio data and the lyrics data paired with the note data is located a minute before the note data; A medium for recording a program for causing a computer to execute a procedure and (b) a procedure of generating audio data in a form of a set of the lyrics data and the note data based on the audio data.