KR20080080013A - Mobile terminal apparatus - Google Patents

Abstract

A portable terminal device is provided to enable a user to pronounce waveform data or tone data written in a user's directly inputted voice signal or a received call signal, according to a pressing action of an operation key, thereby remarkably increasing a degree of freedom for a tone. A voice signal converted into a digital signal by an A/D(Analog to Digital) converter(11) by being inputted from a microphone(10) becomes waveform data(22) in a sampler(21). The waveform data is registered as tone data by a sound source unit(12) by a tone controller(23). When a number key is pressed, the key is detected from a number key allocator(24). An MIDI(Musical Instrument Digital Interface) controller(25) outputs an MIDI message for instructing a sound to be pronounced by a musical interval or tone assigned to the number key, to the sound source unit.

Description

Mobile terminal device {MOBILE TERMINAL APPARATUS}

The present invention relates to a portable terminal device such as a mobile phone or a PHS.

Conventionally, a mobile telephone apparatus having a playing function has been proposed (Patent Document 1).

According to this cellular phone device, while the numeric key, the "#" key, and the "*" key are pressed, the sound source data including the height data of the sound assigned to the key is sent to the sound source, and the sound specified during that time is output. can do. Thereby, the mobile telephone apparatus can be used also as a musical instrument.

Further, portable communication has a function of sampling a voice input from the built-in microphone, a function of creating tone data based on the sampled voice signal, and a function of reproducing long-stream data composed of the voice signal sampled over a long period of time. The terminal is also proposed (patent document 2).

[Patent Document 1] Japanese Patent Application Laid-Open No. 2001-356770

[Patent Document 2] Japanese Patent Application Laid-Open No. 2002-341872

Although the mobile telephone apparatus described in the patent document 1 can be used also as a musical instrument, the tone which can be used is determined in advance, and only a limited tone can be played.

In addition, according to the portable communication terminal described in Patent Document 2, although the tone data created by using the sampled voice can be used, it cannot be used as a musical instrument.

Accordingly, an object of the present invention is to provide a portable terminal device capable of playing using, for example, various tone data created by a user, or reproducing waveform data generated by a user.

In order to achieve the above object, the portable terminal device of the present invention has, for example, a plurality of operation keys and a storage unit in which waveform data is registered, and a sound tone based on waveform data registered in the storage unit. A sound source unit for generating a signal, waveform data creating means for sampling the input audio signal to create waveform data, a registration means for registering the waveform data in the storage unit, and a key for detecting the press and release of the operation key Control means for outputting a message instructing generation of a musical sound signal in a form assigned to the operation key to the sound source unit when the operation detection means and the press of the operation key are detected by the key operation detection means; To have.

A pitch is assigned to each of the plurality of operation keys, and when the control unit detects a press of the operation key, the control means generates the sound signal at the pitch assigned to the operation key to the sound source unit. It may be configured to output a message instructing.

Alternatively, waveform data is assigned to each of the plurality of operation keys, and the control means includes a message for designating a waveform data assigned to the operation key to the sound source unit when a press of the operation key is detected. It may be configured to output a message instructing generation of a sound signal.

Alternatively, the apparatus may further include call means for transmitting and receiving a voice signal with another terminal device to make a call, and the input voice signal may be a voice received from the other terminal device.

Alternatively, the apparatus may further include a call means for transmitting and receiving a voice signal with another terminal device to make a call, and the control means may transmit the sound signal generated by the sound source to the other terminal device via the call means.

Alternatively, the control means may further have a communication means with another terminal device, and the control means may download the waveform data from a predetermined server via the communication means and register it in the storage unit.

Alternatively, the control means may instruct the reproduction of the waveform data registered by the exclusive message of the MIDI format.

According to the portable terminal device of the present invention, the tone data or waveform data created by the user's directly input voice signal or the received call signal can be pronounced as the operation keys are pressed, so that the conventional performance can be performed. Compared to the terminal device, the degree of freedom for the tone is significantly increased.

In addition, an arbitrary word or sentence can be stored as waveform data and sound can be pronounced according to the operation of the operation keys, thereby increasing the range of application of the portable terminal device.

1 is a system configuration diagram of an embodiment of a portable terminal device which is an example of the present invention. Although this portable terminal device is not limited to the portable telephone, a case of the portable telephone will be described here.

In this figure, reference numeral 1 denotes a control unit (CPU: central processing unit) which controls the entire apparatus, and reference numeral 2 denotes various processing programs such as a program for communication and a program for generating sound, various constant data, and the like. ROM (read only memory), code 3 is a random access memory (RAM) which is used as a work area and stores various data, code 4 is a display portion made of a liquid crystal display (LCD), etc., code 5 is a number An operation unit having various operation keys such as a key and a call key, a reference numeral 6 denotes a wireless communication unit including a modulator / demodulation unit, a reference numeral 7 denotes an antenna connected to the wireless communication unit 6, a reference numeral 8 denotes a voice processing unit that encodes and decodes a voice signal, Reference numeral 9 denotes a speaker for a sign language connected to the voice processor 8, reference numeral 10 denotes a microphone for a call that is connected to the voice processor 8, and reference numeral 11 denotes a microphone. An A / D converter (in this example, provided in the voice processing section 8) that converts a voice signal or the like input from the phone 10 into a digital voice signal.

Reference numeral 12 denotes a sound source unit having a plurality of pronunciation channels, reference numeral 13 denotes a D / A converter for converting sound data generated by the sound source unit 12 into an analog tone signal, and reference 14 reproduces the sound signal. It is a speaker. The sound source unit 12 may be a software system by calculation of the control unit 1, or may use dedicated hardware. Although no sound source system is specified, a storage unit for registering waveform data is provided.

Reference numeral 15 denotes a DSP (Digital Signal Processor) for performing a high load process, and reference numeral 16 denotes a bus (data bus and address bus) for transferring data between the components. The DSP 15 is optionally used for high-load processing such as compressing and expanding recorded waveform data into an appropriate format such as adaptive differential pulse code modulation (ADPCM) to effectively use memory capacity. .

In the portable terminal device having such a configuration, the A / D converter 11 converts a voice signal input from the microphone 10, a call signal received from a call partner, etc. into a digital voice signal, and the RAM (3) and the like can be stored. Then, based on the recorded digital audio signal, waveform data or timbre data that can be used in the sound source unit 12 is created, and the waveform data or timbre data is registered in the storage unit of the sound source unit 12, In response to the operation of pressing the operation key of the operation unit 5 by the user, the sound signal of the created tone is pronounced at the pitch assigned to the operation key (operation as a melody instrument) or the tone assigned to the operation key. Data or waveform data can be pronounced (operation as a musical instrument or voice player of a drum timbre). In addition, a signal pronounced by the sound source unit 12 may be transmitted to the call counterpart.

Fig. 2 is a diagram showing a functional configuration of the portable terminal device of the present invention which can execute the above-described operation. In this figure, the microphone 10, the A / D converter 11, the sound source unit 12, the D / A converter 13, and the speaker 14 are the same as those shown in FIG. The sampler module (hereinafter referred to as "sampler") 21, tone control unit 23, number key assignment unit 24, and MIDI (Musical Instrument Digital Interface) control unit 25 are controlled by the control unit 1, respectively. It is realized by software processing. The sound source unit 12 may be a software process as described above, or may use dedicated hardware.

In Fig. 2, a call signal 27 received by the microphone 10 or the wireless communication unit 6 and processed by a voice signal by the voice processing unit 8 is a digital voice signal by the A / D converter 11. Is converted to and input to the sampler module 21.

The sampler 21 creates waveform data (or timbre data) 22 that can be used by the sound source unit 12 using the input digital voice signal as a material. At this time, if necessary, a codec (coder / decoder, DSP 15) for encoding in a compressed data format may be mounted and converted into appropriate compressed data (ADPCM, etc.).

The sampler 21 sets the input digital audio signal for trimming processing or loop reproduction as necessary. Trimming refers to a process of taking out only necessary portions from the recorded waveform data. Loop playback refers to a process of starting playback of the waveform data until the end point is reached, and then returning to the loop point, and then again. A playback method that repeats playback to the end point.

Referring to Fig. 3, the loop point and the end point will be described.

In Fig. 3, reference numeral 30 denotes a digital voice signal (waveform sample), reference numeral 31 denotes a loop point LP, reference numeral 32 denotes an end point EP, and in this example, the position of the loop point 31 is 300 and the termination point. Has a position of 500. When the waveform data is reproduced, playback is performed from 0 (starting point) to 500 (end point), and then returned to 300 (loop point), after which 300 to 500 are repeatedly played. In addition, the trimming is a process in which waveform samples after 500 (end point) are unnecessary for the reproduction process and are deleted.

Thus, the waveform data 22 is created by setting a loop point and an end point, or deleting an unnecessary part with respect to the input digital audio signal as needed. In addition, when creating the timbre data, envelope parameters for giving a desired envelope are also determined.

The generated waveform data (voice data) 22 includes waveform data itself (waveform samples from the start point to the end point), information of the loop point LP and the end point EP, envelope parameters, and the like. .

In addition, the waveform data 22 created in this way is converted into a library and stored in the RAM 3 or the like.

The tone control unit 23 uses the waveform data (tone data) 22 created as described above in the sound source unit 12 to generate a musical sound, so that the waveform data (tone data) in the sound source unit 12 can be generated. It is a program module which performs the process of registering (22). The details of the registration processing of the waveform data will be described later.

The number key assignment unit 24 includes twelve operation keys of the operation unit 5 (in the present embodiment, "0" to "9", "#") so that a user can play using this mobile phone. And number form of " * " As described above, the portable terminal apparatus of the present invention can be used as a melody musical instrument, or as a musical instrument of a drum timbre or a player of voice data. To the number keys are assigned 12 tones (e.g., "degrees" to "hours"), which are the basic sounds of a melody instrument, or tones or waveform data of a drum instrument.

In addition, the setting of whether to operate as a melody musical instrument, a musical instrument of a drum timbre or a voice player, and the setting of which pitch is assigned to each number key or which waveform data is assigned, is performed by the display unit 4 and the operation unit ( 5) can be set by the user.

The MIDI control unit 25 has a function of replacing the number key information of the melody musical instrument or drum musical instrument or voice player assigned to the user interface with a corresponding MIDI message. It issues a note on when the mobile phone's number key is pressed, and issues a note off when it is released. It is also necessary to identify the note on for all the tones, but the identification of the note off depends on the tones.

The sound source unit 12 receives a MIDI message output from the MIDI control unit 25 to generate a musical sound of a corresponding timbre. In addition, a storage unit for storing waveform data or timbre data can be incorporated to change the waveform data or timbre data stored in the storage unit.

The D / A converter 13 converts the digital data output from the sound source unit 12 into an analog signal, which is reproduced by the speaker 14. In addition, this analog signal may be supplied to the transmission side 28 of the wireless communication unit to form a call transmission signal.

4 is a diagram illustrating an example of the configuration of the sound source unit 12.

The sound source unit 12 receives the MlDI message and generates a musical sound of a corresponding timbre. MIDI events include note on / note off, program change (tone change), control change (volume change, etc.), pitch bend (pitch change), and the like. The sound source unit 12 operates accordingly. The sound source output is amplified by the amplifier and made into a real sound by the speaker 14. As shown in Fig. 4, the sound source unit 12 includes a ROM area 41 for storing default tone data, a RAM area 42 for storing waveform data or tone data 22 created by the sampler 21, A plurality of pronunciations (voices) 43 to 46 for sounding a plurality of independent tones and an adder 47 for mixing musical sounds generated by each sounder are provided. The sounders 43 to 46 are provided in the number represented by the maximum number of simultaneous pronunciations of the sound source specification.

Instrument voices are assigned to each of the speakers 43 through 46 by a program change message of MIDI, and each speaker 43 through 46 is pronounced / sounded according to the MIDI note on / note off message. The volume or pitch of each speaker can be manipulated by the message or the pitch bend message. These sounders 43 to 46 are controlled by MIDI messages, and the sound sounded at each sounder 43 to 46 is mixed by the adder 47 to become the final sound.

When the sound source unit 12 is a PCM sound source, pitch EG (envelope generator), filter EG, or amplitude EG can be added as tone data based on previously prepared sampling data (waveform data), Can be processed into various tones to suit the music.

There is a tone map previously defined as a GM (General MIDI) standard, and in the ROM area 41, tone data or sampling data for reproducing the tone on the tone map is stored.

In the RAM area 42, waveform data (tone data) 22 generated by the sampler 21 is stored by the tone control unit 23.

5 is a diagram showing a program change message, a control change message, and an exclusive message according to the present invention.

(a) The program number (tone number) is designated by the program change message, and (b) The bank select number is specified by the bank select message of the control change.

When the waveform control unit 23 registers the waveform data 22 to the sound source unit 12, the tone control unit 23 uses the exclusive messages shown in (c) and (d) to display two types of tone data streams and waveform data streams. Send to (12).

As shown in (c), the tone data sequence includes bank select numbers, program numbers, and various EGs (pitch EG, filter EG, ampli- tual EG) and waveform data sequences constituting the tone parameters. Waveform data IDs for association are included.

As shown in (d), the waveform data string includes a waveform data ID for associating with the tone data stream, a reproduction pitch (pitch for reproduction with C3 of the MIDI key), a sampling frequency, information in waveform data format, and the like. Information and actual waveform data are included.

By sending the exclusive message shown in (c) and (d) from the timbre control unit 23 to the sound source unit 12, it is sent as the exclusive message in the RAM area 42 in the sound source unit 12. The on tone data sequence and the waveform data sequence are stored. As a result, the tone data or waveform data is registered in the sound source unit 12.

In the case of specifying waveform data (voice data), the MIDI control unit 25 specifies the bank select number of the waveform data to be used for the channel to be played, and (b) a bank select message and a program of the waveform data. (A) A program change message specifying the number is transmitted to the sound source unit 12. FIG. As a result, waveform data (or timbre data) used in the channel is specified.

As described above, in the portable terminal apparatus of the present invention, there are two types of operation, an operation as a melody musical instrument (first case) and an operation as a musical instrument or a voice player (drum case) of a drum tone.

Each of these cases will be described in detail.

Fig. 6 is a view for explaining an example of the first case, and Fig. 6A is a pitch assignment to the twelve number keys when utilizing waveform data (voice data) created by a user as a melody tone. Fig. 6B is an external view showing the relationship between the number key of the mobile terminal and the pitch assigned to the number key.

In this case, one type of waveform data (voice data) selected when each number key is pressed is pronounced at the pitch assigned to that number key. In the example shown in FIG. 6, "0" to "9" Twelve notes from "degree" to "hour" are assigned to twelve number keys of "*" and "#".

Fig. 7 is a diagram showing an example of the second case. In this case, waveform data (voice data) is utilized as a drum tone, percussion tone, and sound effect tone. In addition, when assigning different waveform data to each number key, the portable terminal device can be utilized as a voice reproducer.

Fig. 7A is a diagram showing an example of assigning tone data to a number key when utilizing as a drum tone.

In the case of Fig. 7A, each number key becomes individual waveform data. Among these, "Tom" and the like may be simply changed in pitch in the same waveform data.

7B is a diagram showing an example of assigning waveform data to a number key in the case of utilizing as a voice reproducer.

As shown in Fig. 7B, individual waveform data is assigned to each number key, and when each number key is pressed, a voice corresponding to the key is reproduced. For example, when the number key "5" is pressed, it is pronounced "hello", and when the number key "8" is pressed, it is pronounced "yes". In this way, it can also be used as a voice player that pronounces different voices according to the key pressed on the portable terminal device.

Next, the operation of the mobile terminal device configured as described above will be described with reference to the flowchart in FIG. 8. Here, a case of playing in the first case (operation as a melody musical instrument) will be described.

(1) When the user issues a selection instruction 61 on which waveform data (voice data) to play, the system side registers waveform data (voice data) to the sound source unit 12 (S1). . That is, the tone control unit 23 outputs the exclusive message of FIGS. 5C and 5D corresponding to the selected waveform data to the sound source unit 12, Waveform data (voice data) is stored in the RAM area 42.

In addition, in order to pronounce the waveform data, the MIDI control unit 25 designates a channel to be played on the sound source unit 12 (usually, channel 1), so that the waveform data (voice data) to be played by the user is specified. Outputs the (b) bank select message and (a) program change message including the bank select number and the program number.

(2) The user starts playing by pressing or releasing the number key (62). That is, the number key assignment unit 24 always scans which number key is pressed or released (S2).

(3) If there is a key pressed (S3 is YES), the MlDI control unit 25 issues a note on message including a note number corresponding to the pitch assigned to the key (S4).

(4) If there is a released key (S5 is YES), the MIDI control unit 25 issues a note off message including the note number assigned to the key (S6).

(5) Repeat (2) to (4) until the end of the playing (S7).

In this way, the performance using the tone data (waveform data) created by the user can be performed.

Fig. 9 is a diagram for further explaining the operations in each of the first case (operation as a melody musical instrument) and the second case (operation as a musical instrument or a voice player of a drum tone) described above.

Fig. 9A is a diagram showing the first case.

As described with reference to Fig. 8, in this case, the selected tone data (waveform data) is registered in the sound source unit 12 by an exclusive message by the waveform data selection operation 61 of the user. Then, the bank select message and the program change message specifying the bank select number and program number of the waveform data registered for the channel to be played are output to the sound source unit 12.

Then, each time the number key is pressed, a note on message having a note number corresponding to that number key is output, and when released, a note off message corresponding to the number key is output.

Fig. 9B is a diagram showing the second case. As described above, in this case, by pressing the number key, the waveform data (voice data) assigned to the number key is pronounced.

Here, it is assumed that the waveform control unit 23 registers each waveform data (tone data) in the sound source unit 12 as shown in Fig. 9C.

In this second case, when the number key is pressed (62), a bank select message specifying a bank select number corresponding to the waveform data (voice data) assigned to the number key and a program number of the waveform data are designated. A program change message and a note-on message indicating a pronunciation are issued. The waveform data (tone data) used for pronunciation is determined by the bank select message and the program change message, and the pronunciation is instructed from the note on message. As a result, the waveform data (tone data) corresponding to the pressed number key is pronounced. In addition, when a jam of the number key is detected, only the corresponding note-off message is issued.

In addition, in the above, although waveform data was created from the audio signal input from the microphone etc., you may make it acquire from a predetermined server using the communication function which a portable terminal apparatus has.

In addition, in the above, although the microphone 10 for a talk was used for the input of the audio signal at the time of creating waveform data, you may provide a dedicated microphone separately.

In addition, during a call, the audio signal (digital data or analog signal) from the other party may be converted into waveform data. In this way, the voice and voice of the other party can be used as the sampling material for performance.

Note that the RAM area 42 for registering the waveform data does not necessarily need to be in the sound source unit 12, and a predetermined area of the RAM 3 may be used.

Further, the waveform data created as described above can be transmitted to another terminal device using a communication function of the portable terminal device. When in a call state with another telephone terminal device, the voice signal of the waveform data is transmitted by pressing the number key in which the waveform data is registered in advance, so as to convey the will to the other party by voice even in a situation where a conversation cannot be performed. It becomes possible.

Further, the operation in the first case (operation as a melody musical instrument) and the operation in the second case (operation as a musical instrument or a voice player of a drum tone) may be performed at the same time. That is, for example, the sound as a melody instrument is made by the operation keys "1" to "7", and the audio player is made by the operation keys "8" to "9", "*", "0", and "#". The pronunciation as may be made. By doing in this way, more various performance expressions become possible.

In addition, when individual waveform data is assigned to each number key as shown in Fig. 7B, for example, engine sounds such as automobiles, natural sounds such as streams of streams, and the like, such as animal or bird cries, etc. You can also use a variety of sounds. In the above portable terminal device, a data format other than MIDI may be used.

Moreover, in the said portable terminal apparatus, you may make it possible to input various performance symbols etc., in the case of inputting, registering, and playing a musical sound. For example, the forte may press the "1" and "2" keys simultaneously, and the meso piano may be set in advance, such as pressing the "2" and "3" keys simultaneously.

In addition, an image or a moving picture may be displayed on the liquid crystal display device 4 in synchronization with the sound input or reproduction. In addition, a character may be displayed on the liquid crystal display device 4 in accordance with the sound. For example, during the pronunciation of "hello" as shown in FIG. 7B, the characters on the screen may be greeted by greetings. In addition, such an image, a moving picture, a character, or the like may be transmitted to another portable terminal together with the sound.

In addition, the source of the sound is not limited to downloads from a network, recording using a microphone, manual input by a user, and the like. It is also possible to obtain what has accumulated in a home (a personal computer, a DVD, etc.) via the Internet. In addition, you may receive and obtain broadcast waves, such as a television and a radio.

1 is a system configuration diagram of an embodiment of a portable terminal device of the present invention.

2 is a diagram illustrating a functional configuration of an embodiment of a portable terminal device of the present invention.

3 is a diagram for explaining a loop point and an end point.

4 is a diagram showing a configuration example of a sound source unit.

5 illustrates a program change message, a control change message and an exclusive message.

Fig. 6 is a diagram showing an example of assigning pitches to number keys when utilizing waveform data as melody timbres.

Fig. 7 is a diagram showing an example of assigning waveform data to a number key in the case where waveform data is utilized as a drum tone and used for voice reproduction;

8 is a flowchart for explaining the operation of the portable terminal apparatus of the present invention.

9 is a diagram for explaining selection of waveform data.

<Explanation of symbols for the main parts of the drawings>

1: Control unit (CPU)

2: ROM

3: RAM

4: display unit

5: control panel

6: wireless communication unit

7: antenna

8: voice processing unit

9: speaker for sign language

10: microphone

11: A / D Converter

12: sound source

13: D / A Converter

14: Speaker

15: DSP

16: bus

21: sampler module

22: waveform data

23: tone control unit

24: number key assignment unit

25: MlDI control unit

41: ROM area

42: RAM area

43 to 46: Pronunciation (voice)

47: adder

51: pitch EG

52: filter EG

53: Amplitude EG

Claims (7)

A plurality of operation keys, A sound source unit having a storage unit in which waveform data is registered, which generates a sound signal based on the waveform data registered in the storage unit; Waveform data creating means for sampling the input audio signal to create waveform data; Registration means for registering the waveform data in the storage unit; Key operation detection means for detecting the press and release of the operation key; And a control means for outputting a message instructing generation of a sound signal in the form assigned to the operation key to the sound source unit when the key operation detection means detects the press of the operation key. Portable terminal device. A pitch is assigned to each of the plurality of operation keys, And the control means outputs a message instructing the sound source portion to generate a musical tone signal at a pitch assigned to the operation key when the pressing of the operation key is detected. 2. The waveform of claim 1, wherein waveform data is assigned to each of the plurality of operation keys, The control means outputs a message specifying waveform data assigned to the operation key and a message instructing generation of a sound signal to the sound source unit when the pressing of the operation key is detected. A portable terminal device. The apparatus of claim 1, further comprising: call means for transmitting and receiving a voice signal with another terminal device to make a call, The input audio signal is a voice received from the other terminal device. The apparatus of claim 1, further comprising: call means for transmitting and receiving a voice signal with another terminal device to make a call, And the control means transmits the sound signal generated by the sound source to the other terminal device via the call means. The apparatus of claim 1, further comprising a communication means with another terminal device, And the control means downloads the waveform data from a predetermined server via the communication means and registers the waveform data in the storage unit. A portable terminal apparatus according to claim 1, wherein said control means instructs reproduction of said waveform data registered as an exclusive message in MIDI format.

Images