JP4224450B2 - Mobile communication terminal and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a mobile communication terminal, a Karaoke system, and a program that can automatically score singing of a user who sings to a musical piece. <P>SOLUTION: The mobile communication terminal (mobile telephone set 20) includes an audio signal generating means 216, an image signal generating means 217, an external interface means 218, a reference data extracting means 230, a singing data extracting means 231 and a comparing means. The reference data extracting means 230 extracts reference pitch data of a melody of scoring reference based upon musical piece data read out of a data storage means 214. The singing data extracting means 231 extracts singing pitch data based upon an audio signal inputted from an audio input means 215. The comparing means 232 compares the singing pitch data with the reference pitch data. <P>COPYRIGHT: (C)2006,JPO&amp;NCIPI

Description

The present invention relates to a program used Re mobile communication terminal end及 originator such as a mobile phone.

Conventionally, as this kind of karaoke system, a mobile phone as a mobile communication terminal, a portable karaoke device which has a microphone and is connected to the mobile phone, and a radio wave including a sound signal transmitted from the karaoke device are received. What was comprised using the radio | wireless is known (refer patent document 1). In this karaoke system, a sound signal of an accompaniment music played on a mobile phone is sent to a karaoke device connected to the mobile phone. Then, the user's voice signal input from the microphone of the karaoke device is mixed with the sound signal of the music received from the mobile phone, the mixed sound signal is converted into radio waves, and external output to the mobile phone and the karaoke device. Send to the device radio. This radio receives radio waves from the karaoke device and outputs karaoke accompaniment music and user's voice.
JP 2002-341885 A

However, the conventional karaoke system has a problem in that it is necessary to connect a karaoke device, which is a separate device with a microphone provided separately from the mobile phone, to the mobile phone in order to input the user's voice. there were. In this karaoke system, the signal transmitted from the karaoke device connected to the mobile phone to the external output device is only the sound signal and the sound signal of the music, and an external output device having an image display function instead of the radio is externally output. Even when used as an apparatus, there is a problem that it is impossible to display an image for karaoke including lyrics of karaoke music.
Therefore, the present applicant is able to output a karaoke music, sound and image from the mobile communication terminal to an external output device without using a separate device with a microphone different from the mobile communication terminal, and A mobile communication terminal used in the system has been proposed (see Japanese Patent Application No. 2004-1116026).
The present invention relates to an improvement of the above proposed mobile communication terminal and karaoke system, and its purpose is to use karaoke music, sound and images without using a separate device with a microphone different from the mobile communication terminal. Can be output from a mobile communication terminal to an external output device, and a mobile communication terminal, a karaoke system, and a program that enable automatic scoring of a song sung by a user in accordance with music.

In order to achieve the above object, the invention of claim 1 is a mobile communication terminal comprising a voice input means and a communication means for performing communication via a communication network, wherein music data and image data are received. A data storage means for storing, a sound signal of the music is generated based on the music data read from the data storage means, a sound signal of the music and a voice signal of the singing input from the audio input means are mixed, Sound signal generating means for generating a sound signal for output, image signal generating means for generating an image signal for output based on image data read from the data storage means, and output signal generated by the sound signal generating means Sound signal and the output image signal generated by the image signal generating means based on the external interface means for transmitting to the external output device and the music data read from the data storage means A reference pitch data extracting means for extracting the reference pitch data of melodic point reference at a predetermined interval set for the mobile communication terminal, the singing pitch data based on the audio signal input from the voice input means and singing pitch data extracting means to extract, means for obtaining the index of the singing pitch data extracted previously extracted by the singing pitch data extracting means, and the extraction already singing pitch data for the range specified by the index The reference pitch data is acquired , and a set value for the predetermined interval is acquired, and a pitch comparison means for comparing the singing pitch data within the range with the reference pitch data is provided. Is.
According to a second aspect of the present invention, in the mobile communication terminal of the first aspect, in the mobile communication terminal of the first aspect, reference volume data in the grading melody based on the music data read from the data storage means a reference volume data extracting means for extracting, based on the audio signals input from the voice input means, and singing sound volume data extracting means for extracting a singing sound volume data in the predetermined intervals, extracted with the singing sound volume data extracting means Means for obtaining an index of the extracted singing volume data, obtaining the extracted singing volume data and the reference volume data for a range specified by the index, and setting a set value of the predetermined interval get, is characterized in that a sound-comparing means for comparing the said singing sound volume data and the reference volume data in the range
Also, the invention of claim 3, in the mobile communication terminal according to claim 1 or 2, which is characterized in that further comprising means for obtaining information about the status of a device provided in the mobile communication terminal It is.
According to a fourth aspect of the present invention, in the mobile communication terminal of the third aspect , the information relating to the state of the device is volume setting information or echo setting information of the voice input means.
The invention of claim 5 is the mobile communication terminal according to claim 1 or 2, capable of outputting sound output apparatus a voice of the user inputted by the voice input unit, the main body of the mobile communication terminal And a means for receiving a notification that the output destination of the audio signal is changed when the output destination of the audio signal is changed to the external output device or the sound output device in the mobile communication terminal main body. The pitch comparing means compares the singing pitch data with the reference pitch data based on a notification that the output destination of the audio signal has been changed.
The invention of claim 6 is a program executed by a computer provided in a mobile communication terminal comprising voice input means, data storage means, and communication means for performing communication via a communication network. The sound signal of the music is generated based on the music data read from the data storage means, the sound signal of the music is mixed with the voice signal of the singing input from the voice input means, and the sound signal for output is obtained. A function of generating, a function of generating an image signal for output based on image data read from the data storage means, a function of transmitting the sound signal for output and the image signal for output to an external output device; , a function of extracting the reference pitch data of melody scoring criteria based on the read music data from the data storage unit, at predetermined intervals that are set for the mobile communication terminal, the A function to extract the singing pitch data based on the audio signal inputted from the voice input means, a function of acquiring the index of singing pitch data extracted already extracted in the singing pitch data extracting means, designated by the index The extracted singing pitch data and the reference pitch data are acquired for a range to be obtained, the set value of the predetermined interval is acquired, and the singing pitch data in the range is compared with the reference pitch data A program for causing a computer to realize functions.
According to a seventh aspect of the present invention, in the program of the sixth aspect , the computer further realizes a function of acquiring information relating to a state of a device provided in the mobile communication terminal.
In addition, when the output destination of the audio signal is changed to the external output device or the sound output device in the mobile communication terminal body in the program of claim 6 , the program of claim 8 The computer further has a function of receiving a notification that the output destination has been changed and a function of comparing the singing pitch data with the reference pitch data based on the notification that the output destination of the audio signal has been changed. It is characterized by being realized.

The mobile communication terminal according to claim 1 to 5, the sound signal generating means generates a sound signal of music based on the music data read from the data storage means, and the sound signal of the music, it is input from the voice input means The sound signal of the user's voice is mixed to generate an output sound signal. On the other hand, in order to display images corresponding to these music pieces and sounds, the image signal generation unit generates an output image signal based on the image data read from the data storage unit. These output sound signals and image signals are transmitted to an external output device by external interface means. The external output device outputs music based on the output sound signal received from the mobile communication terminal. In addition, when the user utters voice in accordance with the music, the user's voice is output together with the music. Furthermore, the external output device outputs an image corresponding to the music or sound during the output of the music or sound based on the output image signal received from the mobile communication terminal.
Further, in the mobile communication terminal, the reference pitch data extraction means extracts reference pitch data of the grading reference melody based on the music data read from the data storage means . Singing pitch data extraction unit at a predetermined interval set for the mobile communication terminal, to extract the singing pitch data based on the audio signal inputted from the voice input means. And the information of the index of the extracted singing pitch data extracted by the singing pitch data extraction means is acquired. The pitch comparison means acquires the extracted singing pitch data and the reference pitch data for the range specified by this index , acquires the set value of the predetermined interval, and sings the singing pitch data and the reference pitch within the range. Compare the data. Scoring of the user's singing is possible by comparing the reference pitch data of the melody based on the scoring extracted based on the music data in this way and the singing pitch data extracted based on the voice signal of the user's singing become.
The extraction of the reference pitch data and the singing pitch data and the comparison of both pitch data may be performed at an arbitrary timing during the performance of the music piece or after the music performance is completed.
Further, the sound signal generation means is configured to be operable so as not to mix the sound signal of the music generated based on the music data read from the data storage means with the sound signal of the singing input from the sound input means. May be. In the case of this operation without mixing, the singing voice signal input from the voice input means is output from a sound output means such as a speaker provided in the mobile communication terminal body.

In particular, in the mobile communication terminal according to claim 2 , the reference volume data extraction means extracts the reference volume data in the grading melody based on the music data read from the data storage means, and the singing volume data extraction means comprises: Singing volume data is extracted at the predetermined interval based on the audio signal input from the audio input means. Then, the index information of the extracted singing volume data extracted by the singing volume data extracting means is acquired. The volume comparison means acquires the extracted singing volume data and the reference volume data for the range specified by this index , acquires the set value of the predetermined interval, and compares the singing volume data with the reference volume data To do. Scoring reference melody reference volume data extracted based on this song data and singing volume data extracted based on the user's singing voice signal, more accurate scoring of the user's singing Is possible .
Also, by executing a computer provided to the mobile communication terminal program according to claim 6, 7 and 8, the ability to generate sound signals for the output, the ability to generate an image signal for the output, they A function for transmitting a sound signal and an image signal to an external output device, a function for extracting the reference pitch data, a function for extracting the singing pitch data, a function for acquiring the index information, and a function for comparing the pitch data A plurality of functions can be realized.
The program used in the computer may be transferred using a recording medium such as FD or CD-ROM in which the program is recorded as digital information, or may be performed using a communication network such as a computer network.

The “mobile communication terminal” includes a mobile phone such as a PDC (Personal Digital Cellular) method, a GSM (Global System for Mobile Communication) method, a TIA (Telecommunications Industry Association) method, and an IMT (International Mobile Telecommunications) -2000. Standardized mobile phone, TD-SCDMA (MC: Multi Carrier) mobile phone, one of TD-SCDMA (Time Division Synchronous Code Division Multiple Access), PHS (Personal Handyphone System), automobile phone, fixed A telephone having a telephone call function such as a telephone is exemplified. Examples of the “mobile communication terminal” include a communication terminal such as a PDA (Personal Digital Assistance) having a call function in addition to the above telephone.
The “external output device” is a device provided with external interface means, sound output means, and image display means that can receive signals from a mobile communication terminal. For example, analog or digital sound signals and image signals. TV devices and display devices having a general-purpose external interface unit for receiving the images.

Each of the mobile communication terminals includes a data acquisition unit that acquires at least one of the music data and the image data from a server on the communication network that can communicate with the communication unit, and stores the acquired data in the data storage unit. Is preferred. In this case, it is possible to add at least one of music and images that can be output to the external output device by acquiring at least one of music and images from a server on the communication network and storing them in the data storage means.
Each mobile communication terminal further comprises sound processing designating means for designating the content of sound processing to be executed for the music, the voice or both, and the sound signal generating means is the sound processing designating means. It is preferable that at least one sound process of a voice input on / off process, a volume adjustment process, an echo process, a key adjustment process, and a sound quality adjustment process is executed based on the designated designation content. In this case, the user can specify the contents of the sound processing to be executed for the music, the sound, or both by operating the sound processing specifying means. The sound signal generation means is at least one sound process of a voice input on / off process, volume adjustment process, echo process, key adjustment process, and sound quality adjustment process desired by the user based on the designated content designated by the sound process designation means. Execute. By this sound processing, music and voice can be turned on / off from the external output device, and the volume, echo, key, and sound quality of the music and voice can be changed.
Further, in each of the mobile communication terminals, the sound signal generation unit mixes the music and the voice with the message sound so as to superimpose and output the message sound at a predetermined timing during the performance of the music. It is preferable to generate a sound signal. In this case, by outputting the sound signal for output by mixing the music and the voice and the message sound at a predetermined timing during the performance of the music, the message sound is superimposed on the music and the voice and output from the external output device. it can.
Further, in each of the mobile communication terminals, the image signal generation means generates an image signal in which the message is superimposed on the image so that the message is displayed at a predetermined timing during the performance of the music. Is preferred. In this case, by generating an image signal in which the message is superimposed on the image at a predetermined timing during the performance of the music, an image in which the message is superimposed can be output from the external output device.

  According to the present invention, the user's voice uttered in accordance with the music is input using the voice input means of the mobile communication terminal, and the sound signal of the voice and the sound signal of the music are mixed to the external output device. Since it can transmit and output, it is not necessary to use another apparatus with a microphone separately from this mobile communication terminal. In addition, since an image signal can be transmitted to the external output device together with the sound signal and the sound signal of the music from the mobile communication terminal, if an external output device having an image display function is used, the music is output from the external output device. And the user's voice can be output. Furthermore, by comparing the reference pitch data of the melody based on the scoring standard extracted based on the music data and the singing pitch data extracted based on the voice signal of the user's singing, the user's singing can be scored. There is an effect of becoming.

Hereinafter, an embodiment in which the present invention is applied to a karaoke system using a mobile phone as a mobile communication terminal will be described.
FIG. 1 is an explanatory diagram showing a schematic configuration of a karaoke system according to the present embodiment. This karaoke system includes a mobile phone 20 that can communicate via a mobile phone communication network as a communication network, a general-purpose television device 30 that is an image display device with a sound output function as an external output device, and a communication cable 40. I have. The communication cable 40 is a communication cable for sending two left and right audio signals and composite video signals, which are three general-purpose analog signals, from the mobile phone 20 to the television set 30. The composite video signal is a general-purpose signal including all of a luminance signal, a color signal, and a synchronization signal of a television screen. The user operates the mobile phone 20 to start an application program for karaoke, and then selects a desired song from a plurality of karaoke accompaniment songs and performs a start operation. As a result of this start operation, a karaoke image composed of a background image and lyrics contents is displayed on the screen 31 of the television device 30, and accompanying music starts to flow from the speaker 32 of the television device 30. When the user starts to sing along with the accompaniment music, the user's voice is output from the speaker 32 of the television apparatus 30 together with the accompaniment music.

  The mobile phone 20 used by the user 1 in the karaoke system has a configuration capable of executing an application program developed by platform-independent object-oriented programming. Examples of the application program include application programs written in a program language such as JAVA (registered trademark), C, C ++, and the like. The execution environment of these application programs is constructed by middleware such as a JAVA (registered trademark) virtual machine (VM) or BREW (registered trademark). This application program may be preinstalled in the mobile phone 20, or may be downloaded from the download server via the mobile phone communication network 10, stored in the mobile phone 20, and registered.

  FIG. 2 is an explanatory diagram for explaining communication between the mobile phone 20 and a server on the mobile phone communication network 10. An application program download server (hereinafter referred to as “download server”) 11 as a program providing server is connected to the mobile phone communication network 10. In addition to the download server 11, a Web server 12, an information providing server (not shown), a location registration server, a mail server, and the like are connected to the mobile phone communication network 10. These servers may be constituted by a single server. Further, these servers may be configured as dedicated control devices having different configurations, or may be configured using a general-purpose computer system. Each server may be configured by a single computer, or may be configured by connecting a plurality of computers each having a plurality of functions via a network.

When the download server 11 receives a download request from the mobile phone 20, the download server 11 transmits an application program related to the request to the mobile phone 20. In addition, when receiving a browsing request from the mobile phone 20, the Web server 12 transmits a Web page screen that is a browsing screen according to the request to the mobile phone 20. Karaoke data such as karaoke music data and image data used in the mobile phone 20 can be downloaded from the Web server 12 and acquired.
The application program provided from the download server 11 is provided from the application program developer 2. Specifically, for example, it is uploaded from the personal computer on the application program developer 2 side to the download server 11 via a dedicated line or a public line. Note that a recording medium such as an optical disk or a magnetic disk on which the developed application program is recorded is sent from the application program developer 2 to a communication carrier that manages and operates the download server 11, and the application program in the recording medium is downloaded to the download server 11. You may read and provide. The application program thus provided is registered in the download server 11 in a state where it can be downloaded from the mobile phone 20 via the mobile phone communication network 10.

The location registration server is a server that registers and manages information on the current location where each mobile phone 20 is located, that is, location information of a location registration management area where each mobile phone 20 is located. This location registration management area may be an area composed of one cell managed by a base station that communicates wirelessly with each mobile phone 20, or may be an area where a plurality of adjacent cells are managed as one group. Good. A typical size of one location registration management area is several kilometers to several hundred meters. The location registration of each mobile phone 20 with respect to the location registration server is performed periodically when the power is turned on and whenever the cell where the mobile phone 20 is located changes.
The information providing server sends various information such as area information, position information (latitude, longitude, address) of the corresponding position registration management area, content information, and the like through a base station of each position registration management area through a predetermined communication channel. To deliver. As described above, with respect to the communication channel in which various types of information are transmitted, the mobile phone 20 performs a receiving operation at a predetermined timing, thereby acquiring information transmitted through the communication channel and storing it in an internal memory as an information storage unit. Can be saved. The reception operation for each communication channel can be executed at the timing set in advance by the initial setting or at the timing set by the user. This reception operation may be performed regularly or irregularly.
The mail server transfers the mail data transmitted from each mobile phone 20 to the next mail server, receives the mail sent to each mobile phone 20, stores and manages it, and each mobile phone 20 The received mail data is transmitted to each mobile phone in response to a mail acquisition request from the mobile phone.

FIG. 3 is an external view of the mobile phone 20, and FIG. 4 is a schematic configuration diagram showing a hardware configuration of the mobile phone 20.
The cellular phone 20 is a clamshell (folding) type cellular phone, which is an example of a cellular phone, and includes a data storage means including an system bus 200, a CPU 201, a RAM 202, a ROM 203, an input device 204, an output device 205, and a mobile phone. A telephone communication network communication device 206 is provided. Furthermore, the cellular phone 20 includes a memory card drive device 207, an image signal processing device 208, a sound signal processing device 209, and an external interface device 210 as external interface means.

  The components such as the CPU 201 and the RAM 202 exchange various data and instructions of a program described later via the system bus 200. The input device 204 includes a data input key (ten key, * key, # key) 21, a call start key 22, an end key 23, a scroll key 24, a multi-function key 25, a microphone 26, and the like. The output device 205 includes a liquid crystal display (LCD) 27 as a display means, a speaker 28, and the like.

  The mobile phone communication network communication device 206 is configured using a high-frequency transmission / reception circuit for wireless communication, a synthesizer, and the like, and communicates with other mobile phones, the above-described server 11 and the like via the mobile phone communication network 10. Is. The mobile phone communication network communication device 206 exchanges mails with other mobile phones via the mobile phone communication network 10, or external communication networks such as the Internet from the mobile phone communication network 10 via a gateway server. To exchange e-mails over the Internet. Further, the mobile phone communication network communication device 206 downloads an application program provided by the download server 11 or downloads karaoke content provided by the Web server 12 via the mobile phone communication network 10. It is also used as a communication means when receiving distribution information including the position information.

  Further, by operating a key operation unit configured by the above-described numeric keypad 21, call start key 22, call end key 23, and the like that can be operated by the user 1, the user can access the mobile phone 20 with a URL The user can input data such as, start and end a call when receiving a call, and select, start and stop an application program. Further, the user can download and register an application program from the download server 11 by operating the key operation unit, and can select and execute the registered application program. The user can also display information received from the information providing server of the mobile phone communication network 10 by operating the key operation unit.

  The output unit composed of an output device or the like including the liquid crystal display (LCD) 27 and the speaker 28 displays a Web page screen received via the mobile phone communication network 10 on the liquid crystal display (LCD) 27, It is used when notifying the user that information has been received from the mobile phone communication network 10. Specifically, when the information is received, an incoming call notification image is displayed on the liquid crystal display (LCD) 27 of the output unit, or a ringing tone is output from the speaker 28. The output unit is also used when displaying an image or sound related to the execution of the application program such as the game, or when displaying information received from the information providing server. It is done.

  The memory card drive device 207 has a mechanism for making the memory card 29 detachable, and has an interface unit for transmitting and receiving data via a terminal that contacts the electrode terminal of the memory card 29.

  The image signal processing device 208 can be configured by, for example, a single LSI in which analog circuits and digital circuits are mixed, or can be configured by combining circuit components such as a plurality of LSIs. The image signal processing device 208 is controlled by the CPU 201 to generate an image signal used for displaying an image on the display 27 of the output device 205 based on the image data (image file) or to display a plurality of images in a superimposed manner. One image signal is generated from a plurality of image data.

  The sound signal processing device 209 is a device composed of a sound source, an amplifier, an A / D converter, a D / A converter, a mixing circuit, and the like. The sound signal processing device 209 can be configured, for example, by a single LSI in which analog circuits and digital circuits are mixed, or by combining circuit components such as a plurality of LSIs. The sound signal processing device 209 is controlled by the CPU 201 and converts an audio signal input from the microphone 26 of the input device 204 into digital data, or generates an analog sound signal based on digital music data (music file). Or two analog sound signals can be mixed to generate one sound signal.

  The external interface device 210 includes an amplifier, an output connector 210a, and the like. The external interface device 210 includes an image signal (composite video signal) generated by the image signal processing device 208, and a sound signal generated by the sound signal processing device 209 (right and left 2 for stereo). Two audio signals).

FIG. 5 is a functional block diagram of the mobile phone 20 in which the features of the present invention are extracted, and FIG. 6 is an explanatory diagram of the software structure in the mobile phone 20.
The cellular phone 20 includes an application program execution management unit (application program execution management unit) 211, a communication unit 212, a data acquisition unit 213, a data storage unit 214, a voice input unit 215, a sound signal generation unit 216, and an image signal generation unit 217. And external interface means 218. Further, the cellular phone 20 includes a reference data extraction unit 230, a song data extraction unit 231, and a comparison unit 232. Among these means, communication means 212, data acquisition means 213, data storage means 214, audio input means 215, sound signal generation means 216, image signal generation means 217, external interface means 218, reference data extraction means 230, singing data The extracting unit 231 and the comparing unit 232 are provided in the telephone platform 220 on the native side of the mobile phone 20. The comparison unit 232 may be configured not to be provided in the telephone platform 220 but also to be used by the application program execution management unit 211.

  In addition, the mobile phone 20 executes a predetermined program on hardware such as a CPU, whereby an application program execution management unit 211, a data acquisition unit 213, a sound signal generation unit 216, an image signal generation unit 217, an external interface All or part of the functions of the means 218, the reference data extraction means 230, the song data extraction means 231 and the comparison means 232 are realized.

The application program execution management unit 211 includes the system bus 200, the CPU 201, the RAM 202, and the like. This application program execution management means 211 is also called a virtual machine and corresponds to the central “program execution environment” in the software structure of FIG. The application program execution management unit 211 provides software such as a class library, an execution environment management library, and application management used for an application program developed by object-oriented programming, and manages the execution environment of the application program.
Here, the application program can be used by calling a class library such as a function in the program execution environment via a class library API (application interface). The call history of the class library such as this function is held until the virtual execution environment (virtual machine VM) of the application program is terminated.
The execution environment management library in the program execution environment can be used by calling a telephone platform library in the telephone platform described later via the telephone platform API.

The communication means 212 includes the above-described mobile phone communication network communication device 206 and the like, and communicates with other mobile phones, the above-described server 11 and the like via the mobile phone communication network 10.
The data acquisition unit 213 includes the system bus 200, the CPU 201, the RAM 202, and the like. The data acquisition means 213 downloads the karaoke music data and karaoke image data from the cellular phone communication network 10 that can communicate with the communication means 212 or a Web server on the Internet by the user operating the key operation unit. It is acquired and stored in the RAM 202 or the memory card 29.

The data storage unit 214 includes an internal memory such as a RAM 202 and a ROM 203, an external storage device such as a memory card 29, and the like, and an application program such as karaoke music data, karaoke image data, and karaoke acquired by a user from a server or the like. Remember. Examples of files including karaoke song data and karaoke image data include SMAF (Synthetic music Mobile Application Format) files and MMF (Mobile Multimedia Format) files.
In addition to the karaoke song data and karaoke image data, information used for singing may be added to the file such as SMAF. For example, scoring reference melodic data, scoring reference volume data, information for setting a scoring target section (section identification ID, start time, end time), and other additional information may be included. The start time and end time of the scoring target section are set, for example, by the elapsed time (unit: msec) from the beginning of the music.

  The voice input means 215 includes the microphone 26 of the input device 204, and converts the user's voice into an electrical signal and inputs it.

The sound signal generating means 216 is composed of the above-described sound signal processing device 209 and the like. The sound signal generation means 216 generates a sound signal of accompaniment music based on karaoke music data read from the data storage means 214 such as the RAM 202 and the memory card 29. And the sound signal of this accompaniment music and the sound signal of the user's voice input from the sound input means 215 (microphone 26) are mixed, and the sound signal for output sent to the television apparatus 30 is produced | generated.
Further, the sound signal generation means 216 performs at least one sound processing of sound input on / off processing, volume adjustment processing, echo processing, key adjustment processing, and sound quality adjustment processing based on the designation contents designated by the sound processing designation means. Can be executed. When the user designates the designated content, the key operation unit functions as a sound processing designation unit. When the designated content is designated from an application program, the application program execution management unit 211 serves as the sound processing designation unit. Function.

  The image signal generating means 217 is composed of the image signal processing device 208 described above. The image signal generation unit 217 generates an output image signal to be sent to the television device 30 based on the karaoke image data read from the data storage unit 213 such as the RAM 202 or the memory card 29. For example, the image signal generation means 217 generates an image signal composed of lyrics characters corresponding to the accompaniment music in synchronization with the sound signal of the accompaniment music generated by the sound signal generation means 216.

The external interface means 218 includes an external interface device 210 and the like. The external interface unit 218 transmits the output sound signal generated by the sound signal generation unit 216 and the output image signal generated by the image signal generation unit 217 to the external television apparatus 30 via the communication cable 40. .
FIG. 7 is an explanatory diagram of the dedicated communication cable 40. This communication cable 40 has three signal wires and one ground wire, and a connector 41 having a predetermined number of pin terminals is attached to the end portion connected to the output connector 210a of the mobile phone 20. . In addition, a general-purpose pin plug, that is, two pin plugs 42 (red) and a pin plug 43 (white) for a stereo audio signal (L, R), a composite plug, are connected to the end of the communication cable 40 connected to the television set 30. One pin plug 44 (yellow) for video signals is attached.

  The reference data extraction unit 230 is configured by the system bus 200, the CPU 201, the RAM 202, and the like described above. The reference data extraction unit 230 functions as at least a reference pitch data extraction unit. When the volume data is extracted together with the pitch data and used for scoring, the reference data extraction unit 230 functions as both a reference pitch data extraction unit and a reference volume data extraction unit. When the reference data extraction unit 230 functions as the reference pitch data extraction unit, the reference data extraction unit 230 extracts the reference pitch data of the grading reference melody based on the music data read from the data storage unit 214. When the reference data extraction unit 230 functions as the reference volume data extraction unit, the reference data extraction unit 230 extracts the reference volume data in the grading standard melody based on the music data read from the data storage unit 214.

The extraction of the reference pitch data based on the music data can be performed as follows. For example, the SMAF file including the music data designated by the execution instruction from the program execution environment described above and the channel number of the scoring melody in the file are specified. Then, the reference pitch data of the melody based on the scoring is extracted from the channel specified by the channel number. This reference pitch data is extracted for one song along with the integration time, buffered, and returned to the program execution environment. The reference pitch data consists of a combination of a bitmap value (see Table 1) corresponding to a scale of equal temperament set for each note and a pitch bend value (range: -2400 cent to +2400 cent) for that scale. Here, cent is a unit of a pitch obtained by dividing one octave into 1200 equal parts, that is, a unit of a pitch obtained by dividing a semitone into 100 equal parts. The accumulated time is an elapsed time (unit: msec) from the beginning of the music set for the start point of each sound. Note that the reference volume data of the grading reference melody may be extracted together with the reference pitch data from the SMAF file including the music data.

  The singing data extracting means 231 includes the system bus 200, the CPU 201, the RAM 202, and the like. This song data extraction means 231 functions as at least a song pitch data extraction means. When the volume data is extracted together with the pitch data and used for scoring, the singing data extracting unit 231 functions as both a singing pitch data extracting unit and a singing volume data extracting unit. When the singing data extracting unit 231 functions as the singing pitch data extracting unit, the singing data extracting unit 231 extracts the reference volume data in the grading standard melody based on the music data read from the data storage unit 214. Moreover, the song data extraction means 231 extracts song volume data based on the audio | voice signal input from the audio | voice input means 215, when functioning as a song volume data extraction means.

The extraction (pitch detection) of singing pitch data based on the audio signal can be performed as follows. For example, pitch detection is started in response to an execution instruction from the program execution environment described above, and pitch (frequency) data as singing pitch data to be scored is extracted from an audio signal and buffered at regular sampling intervals. This pitch (frequency) data can be extracted by, for example, Fourier transforming waveform data of an audio signal acquired continuously for a predetermined time. Then, the pitch detection is terminated in response to an execution instruction from the program execution environment. The singing pitch data (pitch data) is preferably extracted in a frequency range of about 100 Hz to 1 kHz with 1/16 Hz as a unit. The frequency error of the extraction is desirably 20 cents or less (an error of about 1 Hz when 103 Hz).
At the start and end of the singing pitch data extraction (pitch detection), an event is notified to the program execution environment that is executing the karaoke application.
Also, volume data that is the volume of the singing voice may be extracted (detected) from the voice signal together with the singing pitch data. This volume data is extracted and buffered by 32 relative values equally allocated in the range from 0 (minimum) to 31 (maximum).
If extraction (detection) of the singing pitch data or the like fails, an invalid value (see Table 2) set in advance for each error type is buffered.

  The singing pitch data extracted and buffered as described above is the program execution that executes the karaoke application at any timing during the extraction (detection) of the song data or after the extraction (detection) of the song data. Can be obtained from the environment.

  The comparison means 232 includes the system bus 200, the CPU 201, the RAM 202, and the like. The comparison unit 232 functions as at least a pitch comparison unit, and functions as both a pitch comparison unit and a volume comparison unit when comparing volume data as well as pitch data. When the comparison unit 232 functions as a pitch comparison unit, the comparison unit 232 compares the reference pitch data with the singing pitch data. Further, when the comparison unit 232 functions as a volume comparison unit, the comparison unit 232 compares the reference volume data with the singing volume data. Based on the results of these comparisons, the singing score is calculated.

In addition, the cellular phone 20 of the present embodiment includes a main control unit as control means (not shown). The main control unit exchanges control commands and various data with the application program execution management unit 211 and other units, and performs control in cooperation with them. Further, the main control unit is in the lowest “telephone platform” in the software structure of FIG. 6, and executes a control program and user interface for controlling the above telephone communication and provides a telephone platform library. Or Table 3 exemplifies a class interface (API) as a telephone platform library related to a karaoke application developed by the above object-oriented programming. Tables 4 to 12 illustrate methods or fields of each class interface (API) in Table 3.

For example, “KaraokeData” is a class interface (API) necessary for playback and scoring of SMAF files with karaoke information, and for karaoke such as a method for acquiring melodic information as a reference for scoring as shown in Table 4 Add a method to the class that handles SMAF files (... SmafData).

Similarly, “KaraokePlayer” is a class interface (API) required for playback and scoring of SMAF files with karaoke information. The volume and HV (Humanoid Voice) part of any channel as shown in Table 5 Add a method for karaoke, such as a method for obtaining volume, to the SMAF file playback class (... SmafPlayer). The “HV part” in Table 5 is an audio output part that is processed as if a person uttered using melody data.

Similarly, “ReferenceEvent” is a class interface (API) necessary for reproduction and scoring of a SMAF file with karaoke information, and melodic information (Reference) that serves as a scoring standard such as accumulated time as shown in Table 6 The field etc. which hold various event information which constitutes are prepared.

“ReferenceScore” is a class interface (API) that provides an access means to melodic information (Reference) as a scoring standard in a SMAF file with karaoke information. A field for holding melody information (Reference) is prepared.

“KaraokePlayerListener” is a class interface (API) for implementing a listener that listens for SMAF user events generated by KaraokePlayer, such as methods called when the status of the karaoke player changes as shown in Table 8 Add the karaoke method to the SMAF file playback class (... SmafPlayer).

“MicControl” is a class interface (API) that handles voice input. As shown in Tables 9 and 10, fields indicating whether voice input is valid / invalid are prepared.

“PitchScanData” is a class that holds frequency and audio information obtained by scanning an audio signal input from an audio input, and stores frequency and volume values detected from the audio signal as shown in Table 11. A field for holding array data to be prepared is prepared.

“SpeakerStateListener” is a class that provides a function for receiving a notification that the audio output destination has been changed when the audio output destination is changed. The audio output destination as shown in Table 12 is changed. There are methods to be called when By calling this method, the application can receive a notification that the audio output destination has been changed to a mobile phone main body or a device other than the mobile phone main body (such as a television device, headphones, or earphones).
Since the microphone and speaker of the mobile phone body are close to each other, when pitch detection is performed while music is being output from the speaker of the mobile phone body, the sound emitted from the speaker along with the target sound (singing voice, etc.) May cause trouble in pitch analysis.
Therefore, by using the above "SpeakerStateListener", the user finishes scoring at the timing when the audio output destination is changed to the mobile phone body, and displays text, images, etc. on the screen and sounds a warning sound. Applications that prompt the user to switch the audio output destination to a device other than the mobile phone main unit (such as a TV set, headphones, earphones, etc.), or exclude the pitch detection data in the section until the next switching to a device other than the mobile phone main unit. The creator can take any action.

“DeviceControl” is a class that provides a function for controlling a device unique to a device terminal, and a method for acquiring information of an audio output destination as shown in Table 13 is prepared. By calling this method, the application can actively check whether the audio output destination is set to a device other than the mobile phone body or the mobile phone body from the existing interface (getDeviceState ()).

  In addition, the telephone platform executes various processes in the application program by sending events to the execution environment management library in the program execution environment, or performs application management in the program execution environment via the application management API. The software can be called and used.

  In the case of the clamshell (folding) type mobile phone 20 as in the present embodiment, the phone platform continues to execute the karaoke application even if the mobile phone 20 is folded and closed while the karaoke application is being executed. You may control as follows. However, it is preferable to control the behavior so as to be different from a so-called resident type application only by continuing execution of the karaoke application. In other words, when the mobile phone 20 is folded and closed while a karaoke application is being executed, settings that are valid for a resident application such as a start start setting, a pause transition setting, a timer event, and a power saving mode are ignored. It is preferable to control.

  A control program for constructing a telephone platform for operating the mobile telephone 20 according to a predetermined procedure is stored in the RAM 202 and the ROM 203. Further, a basic OS (operating system) program, a program for constructing the program execution environment, and an application program are also stored in the RAM 202 and the ROM 203. These programs are called up and executed in the work area in the CPU 201 or RAM 202 as necessary.

When downloading the application program from the download server 11 in the mobile phone 20 having the above configuration, the user 1 accesses the download server 11 by operating a key of the key operation unit. As a result, a download selection screen for selecting a downloadable application program is displayed on the display 27. Then, on the download selection screen, the application program desired by the user is selected using the scroll key 24, and when the multi-function key 25 is pressed, the main control unit controls the communication device 206 for mobile phone communication network, The application program is downloaded from the download server 11.
More specifically, a JAVA (registered trademark) application program that can be started by the mobile phone 20 of the present embodiment is based on a JAD file (text data) that is property information and a JAR file (binary data) that is a program body. Become. This program body may be called “MIDlet”, and the property information may be called “MIDlet attribute”. The JAD file includes application data, application version, application vendor name, basic data such as URL data where the JAR file is stored, and whether the application program has already been authenticated, or the application program is connected to the network. This includes setting data such as whether the application program is used in a state in which the application program is used or whether the application program is resident. The JAD file also includes the URL (distribution source address data) of an official Web page that publishes related information related to the application program. When the application program desired by the user is selected on the download selection screen, only the JAD file is downloaded from the application program. Thereafter, the main control unit 210 reads URL data in which the JAR file is stored from the JAD file, accesses the URL, and downloads the JAR file. The application program composed of the JAD file and the JAR file downloaded in this manner is stored in the RAM 102 by the main control unit 210.

  It should be noted that MIDlet-Karaoke, MIDlet-Resident, and MIDlet-Aux for specifying a karaoke control method are set as optional MIDlet attributes in the JAD file of the karaoke application that can be activated on the mobile phone according to the present embodiment. Among these MIDlet attributes, MIDlet-Karaoke is set with a setting value for designating the activation method of the karaoke application. The karaoke application is started based on the set value of MIDlet-Karaoke. For example, when the set value of MIDlet-Karaoke is “Y”, the karaoke function is activated as a terminal-controlled karaoke application that is mainly controlled on the telephone platform 220 side. When the setting value of MIDlet-Karaoke is “N”, the karaoke application is started as a normal JAVA (registered trademark) application. The karaoke function is started as a terminal-controlled karaoke application that is controlled mainly on the JAVA (registered trademark) execution environment (virtual machine) side.

  When executing the downloaded application program, the user 1 operates a key of the key operation unit to display an application selection screen for selecting an application program to be executed on the display 27. On the application selection screen, when the user selects an application program desired by the scroll key 24 and presses the multifunction key 25, the application program execution management means (application program management unit) 211 reads the JAR file. To start the application program. At this time, if necessary, the setting data of the JAD file is referred to and an environment for performing the processing operation according to the setting data is set. Further, when a specific processing operation is performed during execution of the application program, the setting operation of the JAD file related to the processing operation is referred to perform a processing operation suitable for the application program. There are various application programs such as application programs for operating karaoke, games, etc. by operating various keys of the key operation unit, and resident application programs for displaying a standby screen made up of three-dimensional video on the display 27. Application programs are included.

FIG. 8 is a flowchart showing an example of a procedure when a user performs karaoke using the karaoke system configured as described above.
First, the user obtains a karaoke data file (for example, a SMAF file) composed of karaoke music data and karaoke image data by downloading from the Web server 12 or the like, and saves it in the internal memory or the memory card 29 of the mobile phone 20 ( Step 1).
Next, the user connects the mobile phone 20 and the television device 30 with the dedicated communication cable 40 (step 2), and displays a list of application programs on the display 27 of the mobile phone 20. Then, a karaoke application program (karaoke player) is selected from the list displayed on the liquid crystal display 27 and activated (step 3). Processing for turning off the output from the speaker of the mobile phone 20 when starting the application program, processing for preventing the application program from stopping even when the mobile phone body is closed, and specific operation of the application program for karaoke And a process of notifying the user with a message.
Next, the user clicks and selects a karaoke song name to be output from a list of a plurality of karaoke song names displayed on the display 27 during execution of the application program (step 4). As a result of the music selection operation, a karaoke image corresponding to the selected karaoke song is displayed on both the screen of the display 27 of the mobile phone 20 and the screen 31 of the television device 30, and the karaoke song data of the selected karaoke song ( SMAF file) is read into the work memory of the sound signal generation means 216 (sound signal processing device 209), and karaoke can be started.
Next, the user starts karaoke by pressing a predetermined key on the key operation unit (step 5). By this karaoke start operation, the cellular phone 20 starts transmitting predetermined sound signals and image signals to the television device 30 as follows. That is, the mobile phone 20 generates a sound signal of the accompaniment music based on the karaoke music data (SMAF file) and transmits the sound signal to the television set 30. Based on the karaoke image data, the mobile phone 20 generates a karaoke image including lyrics synchronized with the accompaniment music. Generation and transmission to the television device 30 are started. As a result, the selected karaoke accompaniment music flows from the speaker 32 of the television apparatus 30, and a karaoke image including lyrics as shown in FIG. 9 is displayed on both the screen 27 of the mobile phone 20 and the screen 31 of the television apparatus 30. Is done. When the user starts to sing along with the karaoke accompaniment while watching the karaoke image, the user's voice is mixed with the sound signal of the accompaniment tune to generate an output sound signal. Thereby, a user's singing voice begins to flow from the speaker 32 of the television apparatus 30 with the music of an accompaniment.

FIG. 10 is a flowchart showing an example of the processing procedure of the scoring function that can be executed when the karaoke application is activated and the user is performing karaoke. Note that the reference pitch data, reference volume data, and the like used for singing the singing are extracted from a file such as SMAF with karaoke information when the karaoke application is activated and stored in a predetermined area of the memory.
First, when the user performs a karaoke, when the scoring function is started while the karaoke application is being executed, the user's singing pitch and volume at a predetermined sampling interval are detected. At each sampling timing, the waveform of the user's voice signal is measured for a predetermined time, and the measured data is stored in a predetermined area of the memory (step 1).
Next, based on the measurement data of the waveform of the audio signal, the user's singing pitch data and singing volume data are extracted by data processing such as Fourier transform (step 2). This singing data (pitch, volume) data is repeated by the number of times of sampling within a predetermined scoring period, and each is stored in a predetermined area of the memory.
Next, when a predetermined scoring period has elapsed, the reference data (pitch, volume) and the singing data (pitch, volume) are respectively compared, and a score is calculated using a predetermined calculation algorithm ( Steps 3, 4).
Next, the calculated score information is transmitted to the television apparatus 30 and output (step 5).

As described above, according to the present embodiment, since the user's voice uttered in accordance with the music can be input using the microphone of the mobile phone 20, it is not necessary to use a separate device equipped with the microphone. As a result, the circuit configuration can be made without overlapping voice input devices, so that circuit resources can be saved and power consumption can be saved. Moreover, an image for karaoke can be output from the external television device 30 together with karaoke accompaniment music and user's voice. Further, by comparing the reference pitch data of the scoring melody extracted based on the music data and the singing pitch data extracted based on the voice signal of the user's song, the user's singing can be scored. .
Further, according to the present embodiment, not only the pitch data but also the reference volume data and the user's singing volume data are compared, so that the user's singing can be scored with higher accuracy. Note that not only the pitch and volume but also the reference rhythm and the singing rhythm may be compared, and the singing may be scored based on the degree of agreement between the two.
Further, according to the present embodiment, image data including scoring information calculated based on the result of the comparison can be transmitted to the external television device 30 and output, and scoring information output from the television device 30 Can be enjoyed by informing people who enjoy karaoke with their users. This scoring information may be transmitted to the television device 30 as an audio signal and output.
Moreover, according to this embodiment, since at least one of the karaoke accompaniment music and karaoke image which can be output to the television apparatus 30 can be downloaded from the server which manages karaoke data collectively and can be added to the mobile phone 20, karaoke The degree of freedom in selecting a song increases, and it becomes easier to handle new songs.
In addition, according to the present embodiment, by adopting wired communication using the communication cable 40 for communication between the mobile phone 20 and the television device 30, wireless communication using a communication medium such as radio waves and light is possible. On the other hand, there is no need to provide a device that converts electrical signals into radio waves or light, or devices that convert radio waves or light into electrical signals, saving circuit resources necessary for the conversion and simplifying the system configuration. Can be planned. In addition, when outputting a karaoke audio signal or image signal to an external device such as a television device, the frequency band of the audio signal or image signal is transmitted using the communication medium such as the above-mentioned radio wave or light. Therefore, it is not necessary to convert to the frequency band of the signal, so that circuit resources necessary for the conversion can be saved.
Further, according to the present embodiment, since the signal output from the mobile phone 20 is a general-purpose stereo audio signal and composite video signal generally used as an external input signal of the television apparatus, the mobile phone 20 You can enjoy karaoke easily by combining with a general TV set. In addition, since the karaoke accompaniment music, audio sound signal and karaoke image signal transmitted from the mobile phone 20 can be sent to the television device 30 and output, the karaoke accompaniment is different from the case of outputting to the display or speaker of the mobile phone. Tunes and voices can be output at a high volume, and karaoke images can be displayed on a large screen.

  When the music data corresponds to a plurality of song melodies such as a main melody (guide melody) and a sub melody (harmony), the melody subject to scoring may be selected from those song melody. For example, the user operates a key operation unit as a scoring melody designating unit to designate a scoring melody from a plurality of singing melodies. The above-mentioned reference data extraction means 230 of the mobile phone extracts reference data (pitch, volume) of the scoring reference melody specified by the user through the key operation unit. Thus, by extracting the reference data (pitch, volume) of the melody specified by the user, the singing can be scored for a desired melody arbitrarily selected by the user from a plurality of singing melody.

In the karaoke system of the above embodiment, sound signal-related additional functions such as voice input on / off processing, voice external output on / off processing, volume adjustment processing, echo processing, key adjustment processing, and sound quality adjustment processing may be provided. The processing of this additional function can be executed by the sound signal generation means 216 of the mobile phone 20.
For example, the voice input from the microphone (handset microphone) 26 is turned on / off based on the specified content input by the user operating the key operation unit or the command (specified content) generated by the application program, (The microphone for the handset) 26 may be used to turn on / off the sound input to the external device, adjust the volume of the user's voice, or adjust the volume of the karaoke accompaniment music.
Also, on / off the process of applying an echo effect to the user's voice and karaoke accompaniment based on the specified content entered by the user operating the key operation unit or the command (specified content) generated by the application program. Or the degree of the echo effect (echo delay time) may be adjusted. The echo delay time is preferably in the range of 60 msec to 400 msec.
Alternatively, the key of the karaoke accompaniment song may be changed based on the specified content input by the user operating the key operation unit or a command (specified content) generated by the application program.
Also, the sound quality of the user's voice or karaoke accompaniment may be adjusted based on the specified content input by the user operating the key operation unit or a command (specified content) generated by the application program. For example, adjust the sound quality so that the voice of a male user becomes a high-frequency sound like a woman or a child, or conversely adjust the sound of a woman or a child to a low-frequency sound like a man. By doing so, you may produce karaoke.

Moreover, in the said embodiment, you may provide the following additional functions other than the said additional function regarding a sound signal.
For example, as shown in FIG. 12A, a message such as a word “Gambare!” Or the like that cheers the user at a predetermined timing while the user is singing is displayed on the screen 31 of the television device 30 or the display of the mobile phone 20. 27 may be displayed. Further, at a predetermined timing while the user is singing, a message voice such as a word “Gambare!” That cheers the user may be output from the speaker 32 of the television device 30.
Also, as shown in FIG. 12B, a plurality of karaoke songs to be sung by the user are registered in advance and a list of reserved songs for the karaoke is displayed on the screen 31 of the television device 30 or the display of the mobile phone 20. 27 may be displayed. This karaoke song reservation can be input by the user operating the key operation unit of the mobile phone 20.
Also, as shown in FIG. 12C, an image designated by the user by operating the key operation unit or an image designated by the application program as a background image during the karaoke music selection is displayed on the screen 31 of the television apparatus 30. Alternatively, it may be displayed on the display 27 of the mobile phone 20.

In the above embodiment, the case where wired communication using the communication cable 40 is adopted as communication between the mobile phone 20 and the television device 30 has been described, but short-range wireless communication such as Bluetooth (registered trademark), infrared communication, or the like. The wireless communication may be adopted. In this case, external interface means for receiving radio waves and infrared rays from the mobile phone 20 and performing wireless communication with the mobile phone 20 is provided on the external output device side such as a television device.
In the above-described embodiment, analog audio signals and image signals are transmitted from the mobile phone 20 to the television apparatus 30. However, digital audio signals and image signals may be transmitted.
Moreover, although the said embodiment demonstrated the case where the television apparatus 30 was used as an external output apparatus, you may use the external output apparatus which can output a sound and an image other than a television apparatus.
In the above embodiment, a mobile phone that can execute an application program developed by platform-independent object-oriented programming using a program language such as JAVA (registered trademark) has been described, but the present invention is executed by a mobile phone. The present invention can be applied without being limited to the type of application program, and the same effect can be obtained.
Further, the present invention can be applied to a portable PDA in addition to a telephone such as a PHS or a car phone as long as it can execute an application program, and the same effect can be obtained.

Explanatory drawing which shows schematic structure of the karaoke system which concerns on embodiment of this invention. Explanatory drawing which shows communication with the mobile telephone and server of the karaoke system. External view of the mobile phone. The schematic block diagram which shows the hardware constitutions of the mobile phone. The functional block diagram which extracted and showed the characteristic part of the mobile phone. Explanatory drawing of the software structure in the mobile phone. Explanatory drawing of the communication cable used with the karaoke system. The flowchart which shows an example of the procedure when a user performs karaoke using a karaoke system. Explanatory drawing which shows an example of a karaoke image. The flowchart which shows an example of the process procedure of a scoring function. Explanatory drawing of the karaoke image containing a scoring result. (A)-(c) is explanatory drawing of another karaoke image.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Mobile phone communication network 11 Download server 12 Web server 20 Mobile phone 30 Television apparatus 40 Communication cable 211 Application program execution management means 212 Communication means 213 Data acquisition means 214 Data storage means 215 Audio input means 216 Sound signal generation means 217 Image signal generation Means 218 External interface means 220 Telephone platform 221 Application program execution management means 230 Reference data extraction means 231 Singing data extraction means 232 Comparison means

Claims (8)

A mobile communication terminal comprising voice input means and communication means for performing communication via a communication network,
Data storage means for storing music data and image data;
A music sound signal is generated based on the music data read from the data storage means, and the music sound signal and the singing voice signal input from the voice input means are mixed to generate an output sound signal. Sound signal generating means for
Image signal generation means for generating an output image signal based on the image data read from the data storage means;
External interface means for transmitting the output sound signal generated by the sound signal generation means and the output image signal generated by the image signal generation means to an external output device;
Reference pitch data extracting means for extracting reference pitch data of a grading reference melody based on music data read from the data storage means;
At a predetermined interval set for the mobile communication terminal, and singing pitch data extracting means to extract the singing pitch data based on the audio signal input from the audio input means,
Means for obtaining an index of the extracted singing pitch data extracted by the singing pitch data extracting means;
The extracted singing pitch data and the reference pitch data are acquired for the range specified by the index , the set value of the predetermined interval is acquired, and the singing pitch data and the reference pitch data in the range are acquired. A mobile communication terminal comprising pitch comparison means for comparing The mobile communication terminal according to claim 1, wherein
Reference volume data extraction means for extracting reference volume data in the scoring reference melody based on music data read from the data storage means;
Based on the audio signals input from the voice input means, and singing sound volume data extracting means for extracting a singing sound volume data in the predetermined interval,
Means for obtaining an index of the extracted singing volume data extracted by the singing volume data extracting means;
The extracted singing volume data and the reference volume data are acquired for the range specified by the index , the set value of the predetermined interval is acquired, and the singing volume data and the reference volume data within the range are acquired. mobile communication terminal, characterized in that a sound-comparing means for comparing and. The mobile communication terminal 請 Motomeko 1 or 2,
A mobile communication terminal further comprising means for acquiring information relating to a state of a device provided in the mobile communication terminal. The mobile communication terminal according to claim 3 , wherein
The mobile communication terminal characterized in that the information on the state of the device is volume or echo setting information of the voice input means. In the mobile communication terminal according to claim 1 or 2 ,
A sound output device capable of outputting the user's voice input by the voice input means is provided in the main body of the mobile communication terminal,
When the output destination of the audio signal is changed to the external output device or the sound output device in the mobile communication terminal main body, a means for receiving notification that the output destination of the audio signal has been changed,
The mobile communication terminal characterized in that the pitch comparison means compares the singing pitch data with the reference pitch data based on a notification that the output destination of the audio signal has been changed. A program executed by a computer provided in a mobile communication terminal comprising voice input means, data storage means, and communication means for performing communication via a communication network,
A music sound signal is generated based on the music data read from the data storage means, and the music sound signal and the singing voice signal input from the voice input means are mixed to generate an output sound signal. Function to
A function of generating an image signal for output based on image data read from the data storage means;
A function of transmitting the output sound signal and the output image signal to an external output device;
A function of extracting the reference pitch data of the scoring reference melody based on the music data read from the data storage means;
At a predetermined interval set for the mobile communication terminal, a function to extract the singing pitch data based on the audio signal input from the audio input means,
A function of obtaining an index of the extracted singing pitch data extracted by the singing pitch data extracting means;
The extracted singing pitch data and the reference pitch data are acquired for the range specified by the index , the set value of the predetermined interval is acquired, and the singing pitch data and the reference pitch data in the range are acquired. A program for causing the computer to realize a function of comparing with the computer. In the program of Claim 6 ,
A program for causing the computer to further realize a function of acquiring information related to a state of a device provided in the mobile communication terminal. In the program of Claim 6 ,
A function of receiving a notification that the output destination of the audio signal is changed when the output destination of the audio signal is changed to the external output device or the sound output device in the mobile communication terminal body;
A program for causing the computer to further realize a function of comparing the singing pitch data with the reference pitch data based on a notification that the output destination of the audio signal has been changed.