JP2006277194A - Communication terminal - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To easily edit a background sound or sound effects or the like when preparing electronic mail with voice information attached thereto. <P>SOLUTION: A voice signal collected from a microphone 18 is converted into voice data, and when ten keys in an operation key part 8 are operated, sequence data (performance information) corresponding to the operated keys are generated. In this case, timing information is recorded in the sequence data so as to be synchronized with the voice data. Then, the voice data and the sequence data are stored in one file in an RAM 4, and attached to the electronic mail, and transmitted/received. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a communication terminal suitable for use in creating a so-called voice mail.

  2. Description of the Related Art A technique for transmitting and receiving a voice mail recording a user's voice by attaching a voice file to the electronic mail is known. In particular, in Patent Document 1, when a text format message is uploaded from the transmission-side user terminal to the server, the text format message is converted into voice information based on voice color information in the server, and the voice information is further converted. A technique is disclosed in which a background sound (BGM) or a sound effect is added to the user terminal and then transmitted to the receiving user terminal.

JP 2002-99486 A

However, the above-described technique requires a function for converting a text format message into voice information on the server side, which increases the load on the server. In addition, in the finally generated audio signal, for example, when the background sound or sound effect is unintentional for the transmission side user such as being added to the audio information with a deviation from the desired timing, the transmission side It is also impossible for the user to edit only the background sound and the sound effect while leaving the human voice part.
The present invention has been made in view of the above-described circumstances, and provides a communication terminal that can reduce the load on the server and can send and receive e-mails with voice information attached while freely editing background sounds and sound effects. It is aimed.

In order to solve the above problems, the present invention is characterized by having the following configuration. The parentheses are examples.
In the communication terminal according to claim 1, a microphone (18) for collecting a voice signal, a storage means (4) for storing data, and a voice signal collected by the microphone (18) as voice data. Generating sequence data including event data for instructing the start of reproduction of a musical sound in response to an operation on the operation means (8) Sequence data generating means (2, SP6, SP36) to be performed, and the sound data and the sequence data are synchronized and included in the same file (100) and stored in the storage means (4) (SP16) , SP48), a communication means (14) for sending or receiving an e-mail attached with the file, and reading the audio data from the file An audio signal reproducing means (28) for reproducing the audio signal, and a musical sound signal reproducing means (20, 22, 24, 26) for reading out the sequence data from the file and reproducing the musical sound signal in synchronization with the audio signal. It is characterized by having.
Furthermore, in the configuration according to claim 2, in the communication terminal according to claim 1, the operation means (8) has a plurality of buttons, and different event data is assigned to each button. It is characterized by that.
Furthermore, in the configuration according to claim 3, in the communication terminal according to claim 1, the sequence data generation means (2, SP6, SP36) is configured such that the voice data is created by the conversion means (16). When the operation means (8) is operated at a recording timing that is in the middle of being recorded, an audio signal related to the recording timing of the audio data and a musical sound signal related to the operation content of the operation means (8) Are generated so that the files are reproduced simultaneously.
Furthermore, in the configuration according to claim 4, in the communication terminal according to claim 3, the sequence data generating means (2, SP6, SP36) reproduces an audio signal by the audio signal reproducing means (28). When the operation means (8) is operated at a reproduction timing that is in the middle of being played, an audio signal related to the reproduction timing of the audio data and a musical sound signal related to the operation content of the operation means (8) The file is updated so that and are reproduced simultaneously.

  As described above, according to the communication terminal of the present invention, sequence data can be added while being synchronized with voice data, so that an electronic mail with voice information attached can be transmitted and received while freely editing background sounds and sound effects. it can. Furthermore, since the communication terminal transmits and receives voice mails in a completed state, it is sufficient that the server has a function of transferring data, and the load on the server can be reduced.

1． Hardware Configuration of Embodiment Next, the configuration of a mobile phone according to an embodiment of the present invention will be described with reference to FIG.
In the figure, reference numeral 2 denotes a CPU which controls each part in the cellular phone device via the bus 12 based on a control program stored in the ROM 6. A RAM 4 is used as a work memory for the CPU 2. Reference numeral 8 denotes an operation key unit, which includes a numeric keypad and function keys. Reference numeral 10 denotes a display unit that displays various types of information to the user.

  A communication unit 14 transmits and receives RF signals to and from a base station (not shown). That is, a reception baseband signal is generated by demodulating the RF signal received from the base station, and a transmission RF signal is generated by modulating the supplied transmission baseband signal and transmitted to the base station. Reference numeral 16 denotes an audio processing unit that encodes an audio signal input from the microphone 18 to generate a transmission baseband signal and supplies the transmission baseband signal to the communication unit 14. The audio signal is reproduced and the audio signal is supplied to the mixing unit 17. The mixing unit 17 mixes the audio signal from the switch 34 and the audio signal from the audio processing unit 16 and supplies them to the reception speaker 19. The audio processing unit 16 can also convert the analog audio signal input from the microphone 18 into PCM audio data. The converted PCM audio data is supplied to the CPU 2 via the bus 12.

  Reference numeral 20 denotes a sequence data interface which receives sequence data (performance information such as MIDI) via the bus 12. This sequence data consists of a plurality of event data, and each event data is composed of the content of a musical tone to be generated (pitch, timbre, tone length, volume, etc.) and a playback time at which the event is to be generated. ing. In the sequence data interface 20, these event data are supplied to the FIFO buffer 22 in the order of occurrence time. Reference numeral 24 denotes a sequencer. When the reproduction time of the event data located at the head of the FIFO buffer 22 coincides with the current time, the head event data is read and the event data is supplied to the sound source 26. As a result, in the sound source 26, processing such as generation / silence of a digital musical tone signal is executed based on the event data. The event data read by the sequencer 24 is erased in the FIFO buffer 22, and the order of the second and subsequent event data is incremented by "1".

  An audio interface 28 receives PCM audio data via the bus 12 and supplies the PCM audio data to the audio decoder 29. The audio decoder 29 reads out the PCM audio data every predetermined sampling period and generates a digital audio signal. In the mixing unit 30, the digital audio signal supplied from the sound source 26 and the audio decoder 29 is mixed. A DA converter 32 converts the mixed digital audio signal into an analog audio signal. The converted analog audio signal is output from the ring tone speaker 36 and the earphone jack 38 via the switch 34, or is output from the reception speaker 19 via the mixing unit 17. The switch 34 is configured to automatically select the earphone jack 38 side when the earphone plug is inserted into the earphone jack 38, and to select the ringtone speaker 36 or the receiving speaker 19 otherwise. Has been. Information on whether or not a plug is inserted into the earphone jack 38 is supplied to the CPU 2 via the bus 12. The selection of the ring tone speaker 36 and the reception speaker 19 in the switch 34 is performed by operating the operation key unit 8. The sequence data interface 20, the FIFO buffer 22, the sequencer 24, the sound source 26, the audio interface 28, the audio decoder 29, the mixing unit 30 and the DA converter 32 are configured as one sound source LSI 50.

2． Next, the data format of the SMAF (trademark) file 100 employed in this embodiment will be described with reference to FIG. In FIG. 2, reference numeral 102 denotes a content info chunk, which stores various management information such as content type (incoming melody, karaoke system, CM system), language / character code system of character information, and status. Reference numeral 104 denotes an optional data chunk, which stores the genre name, song name, artist name, etc. of the song as necessary. Reference numeral 110 denotes a score track chunk, which stores music sequence data (MIDI). The score track chunk 110 includes one sequence data chunk 112 and a setup data chunk 114. Here, the sequence data chunk 112 stores sequence data of background sounds and sound effects.

  The setup data chunk 114 stores timbre data used by the sequence data, effect setting state, and the like. An audio data track chunk 120 stores PCM audio data. In the present embodiment, the voice data of the speaker output via the voice processing unit 16 is stored in the audio data track chunk 120. A plurality of score track chunks 110 and audio data track chunks 120 can be provided in one SMAF file 100 as necessary. Therefore, the above-mentioned background sound and sound effect can be stored in another audio data track chunk 120 as other audio data.

  Next, an example of a reproduction timing chart of the SMAF file 100 will be described with reference to FIG. First, two audio data track chunks 120 are provided. In one chunk 120 (audio track A), the voice data of the speaker starting from the reference timing t0 is recorded. The other chunk 120 (audio track B) records sound data such as sound effects. In the illustrated example, two systems of score track chunks 110 are also provided. In the sequence data chunk 112 (sequence track A) belonging to one score track chunk 110, event data E1 to E4 of the sequence data related to the sound effect are recorded, and is one sequence data. In the sequence data chunk 112 (sequence track B) belonging to the other score track chunk 110, sequence data BGM1 and BGM2 related to the background sound are recorded to form one sequence data. In each sequence data, the reproduction start timing is recorded by the elapsed time from the reference timing t0. Thereby, each sequence data is recorded in the SMAF file 100 in synchronization with the audio data of the audio track A. The event data may be data designating sound data such as any sound effect stored in the audio data track chunk 120. In that case, when the sequencer 24 reaches the playback time of the event, the sequencer 24 sends a signal for supplying the corresponding audio data to the audio interface 28 to the CPU 2, so that the sound data such as the sound effect is appropriate. Played at the timing.

3． Operation of the embodiment
3.1. Recording process When a message is newly recorded and the SMAF file 100 is created, the user performs a predetermined recording start operation in the operation key unit 8. When this recording start operation is performed, a recording processing routine shown in FIG. 4 is started. When the process proceeds to step SP2 in FIG. 4, the process waits until the PCM audio data supplied via the microphone 18 and the audio processing unit 16 is accumulated for a predetermined time, and when the data is accumulated for the predetermined time. The accumulated audio data is converted into the format of the audio data track chunk 120 and written in a predetermined area in the RAM 4. Next, when the process proceeds to step SP4, it is determined whether or not the sound / effect button has been pressed. Here, the “sound / effect button” is a numeric keypad included in the operation key unit 8. These numeric keys include, for example, relatively short sound effects such as 1: applause, 2: cheers, 3: animal sounds, 4: fanfare, etc., or sequence data or effects of relatively long music (background sounds). Audio data such as sound is assigned.

  If "YES" is determined in step SP4, the process proceeds to step SP6, where sequence data or audio data corresponding to the sound / effect button is stored in a predetermined area in the RAM 4, and new sequence data is created. . If the pressed sound / effect button corresponds to sequence data such as sound effects, the newly created sequence data uses event data in the sequence data such as sound effects. Created. That is, each event data of new sequence data is created so that the reproduction of the musical sound related to the sequence data such as the sound effect is started at the timing when the sound / effect button is pressed. On the other hand, when the pressed sound / effect button is sound data such as a sound effect, the new sequence data becomes event data for starting playback of the sound data at the timing of pressing the sound / effect button.

  Next, when the process proceeds to step SP8, it is determined whether or not an earphone plug is inserted into the earphone jack 38. If "YES" is determined here, the process proceeds to step SP10, and reproduction of a sound effect or the like corresponding to the sound / effect button is started. Specifically, each event data constituting the sequence data is sequentially supplied to the sequence data interface 20. Accordingly, when these event data are supplied to the sound source 26 via the FIFO buffer 22 and the sequencer 24, a musical sound signal such as a sound effect corresponding to the event data is synthesized here, and the sound effect is converted into a DA converter 32, an earphone, and the like. Sound is emitted from the earphone via the jack 38.

  On the other hand, if “NO” is determined in step SP8, the process proceeds to step SP9, and whether or not the receiving speaker 19 is selected in the switch 34 (received in the operation key unit 8 as a supply destination of the sound effect or the like). Whether or not the speaker 19 is selected) is determined. If "YES" is determined here, the process proceeds to the step SP10. However, when step SP10 is executed via step SP9, sound effects and the like are emitted from the receiving speaker 19 via the DA converter 32 and the mixing unit 17. If no earphone is inserted and the ring tone speaker 36 is selected in the switch 34, the step SP10 is skipped. This is because, in a mobile phone, the microphone 18 and the ring tone speaker 36 are close to each other, and the volume of the ring tone speaker 36 is larger than that of the receiving speaker 19, so that the ring tone is being recorded during recording through the microphone 18. This is because if a sound signal is emitted from the speaker 36 for use, there is a possibility that a wraparound may occur. However, when sound signals are emitted through the earphone or the receiving speaker 19 having a relatively low volume, such a problem is unlikely to occur, and sound effects are emitted only in these cases. . In particular, when a sound effect or the like is emitted through the receiving speaker 19, the user can record a message while listening to the sound effect or the like in the style of making a phone call. Even when sound effects are emitted from the receiving speaker 19, if the above-mentioned wraparound problem occurs, an echo canceller function may be added to erase the wraparound signal. Next, when the process proceeds to step SP12, characters or figures representing the contents such as sound effects related to the sound / effect buttons are displayed on the display unit 10. For example, when “1: applause” is selected as a sound effect, an image of a clapping hand is displayed on the display unit 10 together with a character string “playing applause”. As a result, even when no sound effect is actually emitted, the user can recognize which sound effect or background sound is recorded.

  Next, when the process proceeds to step SP14, it is determined whether or not an end operation has been executed, that is, whether or not a stop button that is a predetermined button in the operation key unit 8 has been pressed. If "NO" is determined here, the process returns to step SP2, and the processes of steps SP2 to SP12 are repeated. On the other hand, if the stop button is pressed, “YES” is determined in step SP14, and the process proceeds to step SP16. Here, the SMAF file 100 is constructed based on the audio data and sequence data created in steps SP2 to SP12. At this time, the audio data is recorded in the audio data track chunk 120, and the sequence data is recorded in the score track chunk 110. Further, the contents of the content info chunk 102 or the optional data chunk 104 are added as necessary.

  Next, when the process proceeds to step SP18, an inquiry message as to whether to attach the SMAF file 100 to the e-mail is displayed on the display unit 10, and the process waits until a user operation is performed. Here, if an operation for attaching to an e-mail is performed, “YES” is determined, and the process proceeds to step SP22, where the SMAF file 100 is attached to the designated e-mail. On the other hand, if it is determined as “NO” in step SP20, the processing is ended as it is.

3.2. Effect Sound Correction Processing Further, when correcting the sound effect or the like of the existing SMAF file 100, the user performs a predetermined sound effect correction start operation on the operation key unit 8. When such an operation is performed, a sound effect correction routine shown in FIG. 5 is started. In the figure, when the process proceeds to step SP30, audio data and sequence data are read from the SMAF file 100 for a predetermined period. Next, when the process proceeds to step SP32, audio data and sequence data are transmitted to the audio interface 28 and the sequence data interface 20, respectively. Next, when the process proceeds to step SP34, it is determined whether or not any sound / effect button has been pressed. If “YES” is determined here, the process proceeds to step SP 36, and event data specifying the sound effect corresponding to the sound / effect button is inserted into the sequence data chunk 112. At that time, the elapsed time from the timing when the correction start operation such as sound effects is performed is given to each event data as timing information. Next, when the process proceeds to step SP38, characters or figures representing the contents of the sound effects are displayed on the display unit 10.

  On the other hand, if the sound / effect button has not been pressed, the process proceeds to step SP40 to determine whether or not a predetermined sound / effect / cancel button has been pressed. If “YES” is determined here, the process proceeds to step SP 42, and the event data of the sound effect being reproduced at the current timing is deleted from the sequence data chunk 112. Next, when the process proceeds to step SP44, contents indicating that the sound effect has been canceled are displayed on the display unit 10 together with characters or figures representing the contents of the sound effect. If neither the sound / effect button nor the sound / effect / cancel button is pressed, the processing in steps SP36, SP38, SP42, and SP44 is skipped.

  Next, when the process proceeds to step SP46, it is determined whether a predetermined end operation is executed in the operation key unit 8 (stop button is pressed) or whether the reproduction of the SMAF file 100 is ended. If the end operation has not been executed and the reproduction of the SMAF file 100 has not ended, the process returns to step SP30. Hereinafter, the processing of steps SP30 to SP44 is repeated until such a condition is satisfied. When the end operation is executed or the reproduction of the SMAF file 100 is ended, the process proceeds to step SP48. Here, the SMAF file 100 is modified as necessary. Next, in steps SP50 to SP54, the same processing as in steps SP18 to SP22 described above is executed. In other words, the created SMAF file 100 is attached to the designated e-mail as necessary, and the processing of this routine ends.

3.3. Reproduction Process Next, a process for reproducing the SMAF file created in the cellular phone of this embodiment or the SMAF file created in another terminal and transmitted to the cellular phone of this embodiment together with the e-mail will be described. When reproducing the SMAF file 100, the user selects an operation mode in which both audio data and sequence data are reproduced while being synchronized, or an operation mode in which only one of them is reproduced. Then, when the SMAF file to be reproduced is selected and a predetermined reproduction start operation is executed by the user, the reproduction routine shown in FIG. 6 is started.

  In the figure, when the process proceeds to step SP60, audio data and sequence data are read from the SMAF file 100 for a predetermined interval. Next, when the process proceeds to step SP62, audio data and / or sequence data, or one of the data, is reproduced according to the operation mode. That is, the data to be reproduced according to the selected operation mode is transmitted to the corresponding side of the audio interface 28 or the sequence data interface 20, respectively. Next, when the process proceeds to step SP64, it is determined whether the end operation is executed or whether the reproduction of the SMAF file 100 has ended. If "NO" is determined here, the processing of steps SP60 and SP62 is repeated, and if "YES" is determined, the present routine is terminated.

Four. As described above, according to this embodiment, various sound effects or background sounds are recorded in the sequence data format in the SMAF file 100, so that a voice mail is created while freely editing the background sounds and sound effects. can do. Furthermore, by recording sound effects and the like in the sequence data format, the data amount of the SMAF file 100 can be reduced. Further, since the sound source originally provided in a terminal such as a mobile phone can be used for generating sound effects and the like, new resources are required and cost increase can be suppressed.

Five. Modifications The present invention is not limited to the above-described embodiments, and various modifications can be made as follows, for example.
(1) In step SP38 of the above-described embodiment, characters or figures representing the contents of sound effects and the like are displayed on the display unit 10. At the same time, sequence data related to the sound effects and the like are displayed on the sequence data interface 20. May be supplied to actually sound the sound effect or the like.

(2) In addition to the SMAF file 100, a file in MPEG4 format such as a captured moving image can be attached to the e-mail. In this case, the SMAF file 100 and the moving image file may be reproduced in synchronization with each other at the terminal that has received the e-mail.

(3) The audio data may be data that has been compression-encoded such as MP3 format or AAC format. In that case, the audio decoder 29 may perform decoding processing.

(4) The music recorded in the SMAF file 100 is not limited to monaural, and may be stereo or multi-channel. In that case, a plurality of ring tone speakers 36 may be provided.

It is a block diagram of the mobile telephone of one Example of this invention. 3 is a file configuration diagram of an SMAF file 100. FIG. It is a reproduction | regeneration timing chart of a SMAF file. It is a flowchart of a recording process routine. It is a flowchart of a sound effect correction routine. It is a flowchart of a reproduction routine.

Explanation of symbols

  2: CPU (sequence data generation means), 4: RAM (storage means), 6: ROM, 8: operation key section (operation means), 10: display section, 12: bus, 14: communication section (communication means), 16: Audio processing unit (conversion means), 18: Microphone, 20: Sequence data interface (musical sound signal reproduction means), 22: FIFO buffer (musical sound signal reproduction means), 24: Sequencer (musical sound signal reproduction means), 26: Sound source (musical sound signal reproducing means), 28: audio interface (audio signal reproducing means), 30: mixing unit, 32: DA converter, 34: switch, 36: speaker, 38: earphone jack, 100: SMAF file, 102: Content info chunk, 104: Optional data chunk, 110: Score track chunk 112: sequence data chunk, 114: Setup data chunks, 120: audio data track chunk.

Claims (4)

A microphone that collects audio signals;
Storage means for storing data;
Conversion means for converting an audio signal collected by the microphone into audio data;
An operation means for receiving an operation from a user;
Sequence data generating means for generating sequence data including event data instructing to start playing a musical sound in response to an operation on the operating means;
File generation means for including the audio data and the sequence data in the same file while synchronizing them, and storing them in the storage means;
A communication means for sending or receiving an e-mail attached with the file;
Audio signal reproduction means for reading the audio data from the file and reproducing the audio signal;
A communication terminal comprising: a tone signal reproducing means for reading the sequence data from the file and reproducing the tone signal while synchronizing with the audio signal.   The communication terminal according to claim 1, wherein the operation means has a plurality of buttons, and different event data is assigned to each of the buttons.   The sequence data generating means, when the operating means is operated at a recording timing that is a timing during which the audio data is being created by the converting means, and an audio signal related to the recording timing of the audio data 2. The communication terminal according to claim 1, wherein the file is generated so that a musical sound signal related to an operation content of the operation means is simultaneously reproduced.   The sequence data generation means, when the operation means is operated at a reproduction timing that is a timing during which an audio signal is being reproduced by the audio signal reproduction means, an audio signal related to the reproduction timing of the audio data 4. The communication terminal according to claim 3, wherein the file is updated so that a musical tone signal related to the operation content of the operation means is simultaneously reproduced.

Images