JP2006221270A - Multitask system and method of mobile terminal device with voice recognition function - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To make improvements to solve the problem of difficulty with multitask operability due to small operation and display parts. <P>SOLUTION: A mobile terminal device includes a voice command registering part 126A for registering voice commands that define the execution of the function of calling the functions of a mobile terminal device; a voice command input button 121 for inputting voice from a microphone as the voice commands when the button is pressed; a voice command registering process part 141C for implementing a voice recognition process on the voice inputted to the microphone when the voice command input button is pressed and for registering the result of recognition in the voice command registering part as a voice command; and a voice command process part 141B for inputting the voice command from the microphone while the voice command input button is pressed, subjecting the inputted voice command to the voice recognition process after the button is released and, if the result of recognition matches the registered voice command and the voice command is accepted, calling the function that matches the accepted voice command and executing the operation of the function. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a multitask system of a mobile terminal device. In particular, the present invention relates to a multitask system for a portable terminal device having a voice recognition function, which improves operability.

It is well known that many portable terminal devices including a cellular phone, a PHS (simple cellular phone), and a PDA (personal digital assistant) have a plurality of functions, telephone calls, and communication means.
It is also well known that it has a multitasking function that simultaneously processes a plurality of functions, calls, and communication means. For example, it is also possible for the user to listen to the time signal of the station number 117 while composing mail.

  However, there are many portable terminal devices that are made compact for convenience of carrying, and the operation unit and display unit are usually small, such as when you want to use multitasking that simultaneously processes multiple functions and multiple communication means. Compared with the case of inputting from a display device or a keyboard used in a desktop type computer of this type, there are many cases where the operation is considerably difficult and there is a problem in use.

The following conventional techniques are disclosed in relation to the problems of the present invention.
Conventionally, in order to perform a hands-free call with a foldable mobile communication terminal without requiring an external device and performing no special operation, a transmitter is provided in the hinge part of the foldable mobile communication terminal, and a receiver is provided in the cover part. When the terminal casing is opened and a normal call is made, a transmitter and a receiver provided on the inside are used, and the terminal includes a sensor that detects that the casing is folded. Sensors that detect whether something is sandwiched between the lid and the body when folded are provided, and when the housing is folded, for example, in a chest pocket of a shirt, these sensors Detected, the transmitting unit and the receiving unit are automatically switched to the transmitting unit and the receiving unit, and there is one that can perform a hands-free call by simply folding the terminal housing and pinching the pocket (for example, Patent Document 1). reference).

However, in the above-mentioned Patent Document 1, the positions of the transmitter and receiver are improved, enabling hands-free calling and improving operability. However, as pointed out in the problem of the present invention, There is no suggestion of improving the difficulty of multitasking operability with a small operation unit and display unit.
In addition, in the past, in order to provide an emergency call reception device that can accurately detect the contents of an emergency and realize accurate processing when an emergency occurs, an emergency from a mobile terminal that reports the occurrence of an emergency by voice or image An emergency call receiving device for receiving a report, the emergency call receiving device comprising voice recognition means for recognizing the voice, the recognition means comprising a database storing voice recognition keywords, based on the database There is one that recognizes the voice, analyzes the contents of the report based on the recognition result, and reports the emergency call to the optimum related department based on the analysis result (for example, see Patent Document 2).

However, in the above-mentioned Patent Document 2, the content of the notification is analyzed by the voice recognition function, and the content of the emergency is accurately detected when the emergency occurs, and the accurate processing is realized. As described above, there is no suggestion of improving the difficulty of multitasking operability by the small operation unit and display unit of the mobile terminal device.
Conventionally, the control device with a voice recognition function according to the present invention is a device that executes a predetermined function based on a voice recognition result in order to make it possible to easily execute various functions by effectively using the voice recognition function. The predetermined function is executed through a plurality of operation procedures by an external operation, and is associated with the execution instruction of the predetermined function so that the predetermined function is directly executed without the operation procedure by the external operation. Registration means capable of registering a predetermined voice command, and when it is recognized that the input voice matches the voice command registered in the registration means, a predetermined function is directly set based on an execution instruction associated with the voice command. It was comprised so that the control means to perform may be provided. According to this configuration, there is one that can effectively use the voice recognition function and can easily execute various functions (for example, see Patent Document 3).

However, in Patent Document 3, various functions are easily operated and executed using the voice recognition function. However, as pointed out in the problem of the present invention, the small operation unit and display unit of the portable terminal device There is no suggestion of improving the difficulty of multitasking due to.
Conventionally, in order to perform fingerprint authentication with a simple operation and increase the security of mobile radio terminals, etc., a fingerprint reader is provided in the power switch of the mobile radio terminal, and fingerprint authentication is performed when the power switch is pressed. If it matches the fingerprint registered in the unit, the mobile radio terminal is operated. Furthermore, the voice print of the voice input from the microphone is checked against the voice print registered in the voice print registration unit, and if it matches, the call can be made. After a certain period of time has elapsed after the call starts, the power is turned off by the timer unit and the operation state remains unchanged. Even if the phone is lost or stolen, unauthorized use can be prevented, fingerprint authentication can be performed simply by pressing the switch, the security of the mobile wireless terminal can be increased, and a multi-stage switch is used for the call switch. Thus, it is possible to perform fingerprint authentication and random number generation by half-pressing and to make a call by fully pressing (see, for example, Patent Document 4).

However, in Patent Document 4, voiceprints are collated and security is improved. However, as pointed out in the problem of the present invention, it is difficult to operate multitasking by a small operation unit and display unit of the mobile terminal device. There is no suggestion of improving sex.
Conventionally, for example, an electronic device with a voice operation function, a voice operation method in an electronic device, and an electronic device with a voice operation function can be easily and reliably performed by a user's voice input, such as making a call from a car. In order to provide a motor vehicle equipped with, voice input means, voice recognition means for recognizing and processing a voice signal input to the voice input means, and data that pre-registers a target to be called and a specific action corresponding to the target The target to be called and operated in the data registration unit is specified by the voice recognized by the registration unit and the voice recognition unit, and then a specific action corresponding to this target is specified by the voice recognized by the voice recognition unit. And a processing means for doing this (for example, see Patent Document 5).

However, in the above-mentioned Patent Document 5, the operation is easily and reliably performed by the user's voice input. However, as pointed out in the problem of the present invention, the small operation unit and the display unit of the portable terminal device are used. There is no suggestion of improving the difficulty of multitasking operability.
In addition, conventionally, it is automatically determined whether to execute the processing content corresponding to the voice command based on the voice information input from the voice input unit or to execute the processing content set separately. A voice recognition dictionary storing corresponding voice information, a voice command table storing voice commands corresponding to the voice information in advance, a voice input unit for inputting voice, and voice information from the inputted voice based on the voice recognition dictionary If the voice information and voice command match, the voice command comparison unit for comparing whether the voice information and voice command in the voice command table match, and the voice information and voice command correspond to the voice command. If the voice command execution unit that executes the processing content does not match the voice information and the voice command, the processing content set separately for the voice information is executed. Comprising a speech information processing unit that (for example, see Patent Document 6).

  However, in the above-mentioned patent document 6, whether or not the input voice information is a voice command is recognized as a voice command, the corresponding processing content is executed in the case of a voice command, and another processing content is executed in the case of not being a voice command. However, as pointed out in the problems of the present invention, there is no suggestion of improving the difficulty of multitasking operability by the small operation unit and display unit of the portable terminal device.

JP 2003-51871 A Japanese Patent Laid-Open No. 2004-23468 JP 2003-5781 A JP 2000-165378 A JP-A-11-142165 JP 2000-242464 A

  Therefore, in view of the above problems, an object of the present invention is to provide a multitask system of a portable terminal device having a voice recognition function, which improves the difficulty of multitasking by a small operation unit and a display unit.

  In order to solve the above problems, the present invention provides a voice command registration for registering a voice command defining a function call of the portable terminal device and execution of the function in a multitask system of a portable terminal device having a voice recognition function. A voice command input button for inputting voice from the microphone of the portable terminal device as a voice command during the pressing period, and voice recognition processing for the voice input to the microphone during the pressing period of the voice command input button A voice command registration processing unit for registering a recognition result as a voice command in the voice command registration unit, and inputting a voice command from the microphone during a pressing period of the voice command input button, and releasing the pressing of the voice command input button. The voice command input later is processed by voice recognition. A voice command processing unit that calls a function corresponding to the received voice command when the voice command registered in the voice command registration unit is received and is received, and performs a function operation; Provided is a multitask system of a portable terminal device having a voice recognition function.

In addition, the operation key of the mobile terminal device and the voice command processing unit independently call the function of the mobile terminal device and execute the function.
In addition, the voice command registration processing unit displays a plurality of functions of the portable terminal device as icons on the display unit, and inputs the selected function to the microphone during the pressing period of the voice command input button. The recognized voice is subjected to voice recognition processing, and the recognition result is registered as a voice command.

The voice command registration processing unit displays a plurality of functions of the portable terminal device as icons on the display unit, displays a selected function name, and guides recording of the pronunciation of the function name, after recording A screen for guiding the registration of the recognition result after the speech recognition processing and a screen for completing the registration are displayed.
When the voice command input button is pressed with respect to the voice input function selected from the icon display displayed on the display unit so that a plurality of functions of the portable terminal device can be selected. A list of voice commands registered in the voice command registration unit is displayed, and a target voice command can be selected and executed.

The voice command processing unit sounds a command acceptance notification sound when a voice command is accepted as a recognition result, and a command non-acceptance notification sound when a voice command is not accepted.
The voice command processing unit has a voice synthesis function. When a voice command is received, the voice command processing unit repeats the received voice command from the speaker of the portable terminal device by voice synthesis.
In addition, the voice command processing unit inputs a voice command from the microphone during a pressing period of the voice command input button, and when a predetermined time has elapsed before releasing the voice command input button, the input voice command Voice recognition processing.

In addition, the voice command processing unit forcibly terminates the operation of the function that is being executed when the voice command input button is silent after the function operation is performed and the voice command is not input and the press is released. Let
The voice command processing unit has a voiceprint analysis function, the voice command registration processing unit C causes the voice command registration unit to register a user's voiceprint input from a microphone, and the voice command processing unit The voice print input from the microphone is analyzed before the recognition process.

The voice command input button is a non-lock type button and is located on the side of the portable terminal device.
The voice command input button is used by selecting one of the ten key, function key, and pointing device of the portable terminal device.
In addition, the voice command processing performs mail, camera, calculator, memo, Internet, dictionary, phone book, schedule, remote control, voice input function call, and function operation.

The functions of the remote control include the functions of a television using infrared rays, a server using radio waves for wireless LAN, a refrigerator using radio waves for BLUETOOTH, and a remote control for home appliances including lighting.
In addition, the multi-task system of the portable terminal device allows the voice command processing unit to call the phone book function and perform the operation processing of the function to listen to the time signal while creating the mail by reading the mail function with the operation key. To do.

  The portable terminal device is foldable, and has a pair of microphones and speakers on the inside and outside, and an open / close detection sensor for detecting opening and closing of the portable terminal device, and the portable terminal device is open. Only the inner pair of microphones and speakers are enabled, and when the portable terminal device is closed, only the outer pair of microphones and speakers is enabled, and the voice command processing unit is open for the portable terminal device. In this case, a voice command is input from the inner microphone, and when the portable terminal device is closed, a voice command is input from the outer microphone.

  Further, the multi-task system of the mobile terminal device reads out a mail function with an operation key and creates a mail while the mobile terminal device is open, while the voice command processing unit calls a television function, The operation of the function to turn on the power is made possible, and the operation of the function of adjusting the volume of the television and switching the channel is performed in the state where the creation of the mail by the operation key is completed and the portable terminal device is closed.

Further, in the multi-task system of the mobile terminal device, the voice command processing unit is configured to call a television function in a state where the mobile terminal device is closed, to operate a function to turn on a TV, adjust a volume, and switch a channel. Perform function operations.
According to another aspect of the present invention, there is provided a multitasking method for a mobile terminal device having a voice recognition function, wherein one of a plurality of functions of the mobile terminal device is selected and the mobile terminal device is operated during a period of pressing a voice command input button. A step of inputting voice from a microphone, a step of performing voice recognition processing on the input voice, and a step of registering the recognition result obtained by the voice recognition processing as a voice command defining calling of the function of the mobile terminal device and execution of the function A step of inputting a voice command to the microphone during the pressing period of the voice command input button, a step of performing voice recognition processing of the voice command input after releasing the pressing of the voice command input button, and voice recognition of the voice command A process of accepting a voice command when the processed recognition result matches a registered voice command; Call the function of the previous term portable terminal device corresponding to the voice command, characterized in that it comprises a step of performing an operation function, provides a multitasking method for a portable terminal apparatus having a speech recognition function.

  As described above, according to the present invention, one of a plurality of functions of the mobile terminal device is selected, voice is input from the microphone of the mobile terminal device while the voice command input button is pressed, and the input voice is voiced. The recognition result obtained by the recognition process and the voice recognition process is registered as a voice command that defines the function call of the mobile terminal device and the execution of the function, and the voice command is input to the microphone while the voice command input button is pressed. Performs voice recognition processing for voice commands that are entered after the command input button is released, and accepts voice commands when the recognition results obtained by voice recognition processing for voice commands match the registered voice commands. Since the function of the mobile terminal device is called and the operation of the function is executed, the small operation of the mobile terminal device is performed. Parts, to improve the difficulty of the operation of multitasking by the display unit. This is because the function execution by the voice command that does not require the screen display and the function execution by the key operation can be executed independently, so that the two functions can be easily used even in a small casing.

In addition, since voice commands can be registered for all functions / operations of the portable terminal device, even in a portable terminal device having multiple functions and a plurality of communication means, the means is taken care of for executing the desired function. Therefore, operability can be improved according to the user's preference.
In addition, since the functions of the user's portable terminal device can be executed / terminated only by a button operation and voice input, it can be easily operated even in the dark.

In addition, since the functions of the user's portable terminal device can be executed with the casing of the portable terminal device closed, the use of screen display, backlight, etc. can be reduced, and power can be saved.
Since the voice print of the voice input by the microphone is analyzed before the voice command voice recognition processing, it is possible to prevent an unauthorized use by a third party and a malfunction due to a neighbor's voice.

Embodiments of the present invention will be described below with reference to the drawings.
FIG. 1 is an overall view illustrating an outline of a multitask system of a mobile terminal device having a voice recognition function according to the present invention.
As shown in the figure, the mobile terminal device 111 includes a plurality of functions, telephone calls, and communication means, and is a foldable type as an example. The mobile terminal device 111 includes an operation key 113, an antenna 118, and a display unit 119. A telephone line / Internet 134 including a mobile phone line and a fixed telephone line is connected via a radio wave 114 to a station to perform a call and communication, and further, a file management, a printer is connected to a server 136 via a radio wave 116 for a wireless LAN. It performs output, connects to the television 135 via the infrared ray 115, performs remote control of the television 135, further communicates with a personal computer (not shown), and uses home appliances such as a refrigerator 137 via the radio wave 117 for BLUETOOTH. Connected to the machine, inventory management, freshness management, etc. of the refrigerator 137 It has, is controlled by a program.

Further, the mobile terminal device 111 has a function for the user to perform multitasking using the operation keys 113, for example, a function for listening to a time signal of the station number 117 while creating a mail.
Further, as will be described in detail below, together with the above multitasking, the present invention allows the user to independently operate the voice input operation function using the voice recognition function of the mobile terminal device 111 and the operation function of the operation key 113. It has a function of executing multitasking, for example, a function of listening to a time signal of the station number 117 by voice input while creating a mail using the operation keys 113.

  Further, the mobile terminal device 111 is provided with a voice command input button 121. The voice command input button 121 is, for example, a non-lock type button located on the side surface of the mobile terminal device 111, and a certain time during the button pressing period. It is possible to input a voice input only as a voice command 112 and use various functions including a call and communication function of the mobile terminal device 111 as a pre-registered corresponding operation. Here, since the voice command 112 executes the operation function of the portable terminal device 111 by voice input using the voice recognition function, the function call of the portable terminal device 111 registered by the user in advance, Is a voice keyword that defines the execution of.

  In particular, in the multitask function of the mobile terminal device 111 described above, a function based on voice recognition is executed for the function execution that does not require the screen display of the display unit 119 of the mobile terminal device 111, and the screen display of the display unit 119 is required. The function by the operation key 113 is executed in response to the function execution, and the simultaneous use of the functions of the portable terminal device 111 having the operation key 113 and the display unit 119 is facilitated to reduce the burden on the user.

FIG. 2 is an external view schematically showing the mobile terminal device 111 of FIG. As shown in FIG. 5A, an inner microphone 122 and an inner speaker 123 are provided as a first system on the inner side of the folding portable terminal device 111 opened, and the inner speaker 123 is the antenna 118 described above. The display unit 119 and the voice command input button 121 are located on the upper side.
Further, the inner microphone 122 is located on the lower side together with the operation key 113 described above.

As shown in FIG. 4B, an outer microphone 124 and an outer speaker 125 are provided as a second system outside the upper side of the foldable portable terminal device 111 where the casing is closed. The outer microphone 124 is positioned below the outer speaker 125 when the casing of the mobile terminal device 111 is changed from the opened state to the closed state.
FIG. 3 is a block diagram showing a schematic configuration of the mobile terminal device 111 of FIG. As shown in the figure, the mobile terminal device 111 is provided with a wireless unit 140, and the wireless unit 140 transmits and receives radio waves 114 and radio LAN radio waves 116 to and from a base station via an antenna 118.

Microphone / speaker system changeover switches 132 and 133 are connected to the radio unit 140. The microphone / speaker system changeover switches 132 and 133 are the first system inner microphone 122, the inner speaker 123, the second system outer microphone 124, and the outer speaker. This is a switch device that connects and disconnects the wireless unit 140 by switching 125.
A control unit 141 is connected to the radio unit 140, and the control unit 141 is configured by a CPU (Central Processing Unit) and controls the entire mobile terminal device 111. In particular, a speaker / microphone switching unit 141A and a voice command processing unit which will be described later. 141B and a voice command registration processing unit 141C.

  A memory 126 is connected to the control unit 141, and the memory 126 stores a control program for the mobile terminal device 111, for example, a program for mail, camera, calculator, memo, Internet, dictionary, phone book, schedule, short-range communication, and voice input. In addition to storing, various data handled by the mobile terminal device 111 such as control data, images, and remote control data are stored, and in particular, a voice command registration unit 126A described later is provided.

Furthermore, a display unit 119 is connected to the mobile terminal device 111, and the display unit 119 is configured by an LCD (liquid crystal display unit). The display unit 119 is an email, a camera, a calculator, a memo, the Internet, a dictionary, a phone book, a schedule, Short distance communication and voice input functions are displayed as icons, and images, mails, etc. are displayed.
Further, an open / close detection sensor 127 is connected to the control unit 141, and the open / close detection sensor 127 is a sensor that detects the open / closed state of the housing of the mobile terminal device 111. When the control unit 141 is in the closed state, the display unit 119 The backlight of the LCD is turned off and the screen of the display unit 119 is not displayed.

Furthermore, a voice command input button 121 is connected to the control unit 141, and the control unit 141 uses the voice input to the inner microphone 122 or the outer microphone 125 as the voice command 112 via the radio unit 140 as described above. The voice command 112 by voice is received when the voice command input button 121 is pressed.
Further, a camera 128 is connected to the control unit 141, and the control unit 141 displays an image captured by the camera 128 on the display unit 119 and stores it in the memory 126.

An operation key 113 is connected to the control unit 141, and the operation key 113 includes a button switch group including a numeric keypad, a keyboard including function keys, and a pointing device.
Further, an Ir (Infrared) input / output circuit 129 is connected to the control unit 141, and the Ir input / output circuit 129 is an input / output circuit for infrared rays 115 for performing infrared communication and infrared remote control.

Further, a BLUETOOTH circuit 130 is connected to the control unit 141, and the BLUETOOTH circuit 130 is a circuit for performing short-distance communication with a radio wave 117 for BLUETOOTH.
Next, the speaker / microphone switching unit 141A of the control unit 141 detects the open / close detection sensor 127 to detect the open / close state of the casing of the mobile terminal device 111, and the microphone / speaker system switch 132, 133 / Speaker system switching signal 131 is output, and the operations of the first system inner microphone 122 and inner speaker 123 and the second system outer microphone 124 and outer speaker 125 are switched.

That is, the first microphone inner microphone 122 and the inner speaker 123 operate in the open state of the casing of the mobile terminal device 111, and the second microphone outer microphone 124 and the outer speaker 1 are operated.
In the closed state of the casing of the portable terminal device 111, the second system outer microphone 124 and the outer speaker 125 operate, and the first system inner microphone 122 and the inner speaker 123 stop operating. To do.

Since the first system (inner side) inner microphone 122, the inner speaker 123, the second system (outer side) outer microphone 124, and the outer speaker 125 are switched depending on the open / close state of the casing of the portable terminal device 111, the casing of the portable terminal device 111 Execution by the voice command 112 is possible not only in the opened state but also in the closed state of the portable terminal device 111.
Furthermore, the voice command processing unit 141B of the control unit 141 uses the voice command input button 121 to analyze the voice command 112 by voice recognition processing on voice input from the inner microphone 122 or the outer microphone 124 via the wireless unit 140. When the voice command 112 analyzed by the voice recognition process matches the voice command 112 registered in advance, the function of the mobile terminal device 111 is called to execute the operation.

Further, the voice command registration processing unit 141C of the control unit 141 uses the voice command input button 121 to analyze the voice input from the inner microphone 122 or the outer microphone 124 via the wireless unit 140 by voice recognition processing. The result is registered as a voice command 112 in the voice command registration unit 126A of the memory 126.
Although the schematic configuration of the portable terminal device 111 has been described with reference to FIG. 3, the LCD of the display unit 119 and the camera 128 are well known to those skilled in the art and are not directly related to the present invention. Detailed description thereof will be omitted.

  FIG. 4 is a schematic diagram for explaining a function display menu displayed on the display unit 119. As shown in the figure, a menu that is a standby screen for selecting a function of the mobile terminal device 111 is displayed on the display unit 119, and icons for each function are displayed in a list in the menu. As an example, mail, camera, and calculator functions are displayed as icons from the upper left side of the icon displayed on the display unit 119, memo, Internet, and dictionary functions are displayed as icons from the left side of the interruption, and the phone book is displayed from the lower left side. The schedule, short-range communication functions, etc. are displayed as icons. A desired icon is selected by operating a cursor surrounded by four corners with the operation keys 113 using the operation keys 113.

A program for operating these functions is stored in the memory 126, and when an icon displayed in a list on the menu is selected with the operation key 113, the operation is executed by the control unit 141. In addition to this, a program for operating the above-described speech recognition function and speech synthesis function is stored in the memory 126 and the operation is executed by the control unit 141.
FIG. 5 is a control flow diagram illustrating the registration of the voice command 112 for the remote control function of the television 135 by the infrared rays 115 in the operation screen example of the voice command registration processing unit 141C of the control unit 141 in FIG. As an example, using this figure, using the remote control function of the television 135 with infrared 115 built in the portable terminal device 111, “TV TV” is registered as a voice command 112 as a voice command 112 “turn on TV power”. A case of registration in the unit 126A will be described.

In step 41, first, the function is displayed on the menu screen of the display unit 119 by using the operation key 113 by the user, thereby enabling selection of an icon for short-range communication.
In step 42, the near field communication menu screen is displayed on the display unit 119 by selecting the near field communication icon. This near field communication menu screen includes functions of infrared communication, infrared remote control, BLUETOOTH, and wireless LAN. The list is displayed in text. Using the operation key 113, the display unit 119 enables selection of character display of the infrared remote controller. If the selection is wrong, the user can press the “Return” key to redo the selection. The same applies hereinafter.

  If character display of the infrared remote control is selected in step 43, the menu screen of the infrared remote control is displayed on the display unit 119, and a list of functions of TV, video, and user settings is displayed on the menu screen of the infrared remote control. Is done. Using the operation key 113, it is possible to select a character display on the display unit 119. The user setting means that if there are functions other than television and video, the user setting can be performed. The same applies hereinafter.

  If the television character display is selected in step 44, the display unit 119 transitions to the execution screen of the television remote controller. In this execution screen, keys such as channel switching (CH +, CH-), volume adjustment (VOL +, VOL-), and power supply (PWR) are displayed according to the screen, and the corresponding key is pressed using the operation key 113. As a result, infrared 115 data can be transmitted to the television 135.

  In Step 45, in this embodiment, since it is desired to register the voice command 112 for the key corresponding to the power source [PWR] of the TV remote controller, the power key “PWR” is set using the operation key 113 while the voice command input button 121 is pressed. Press the corresponding key according to the execution screen to enter the voice command registration mode. In this case, data of the infrared rays 115 due to the pressing of the power “PWR” key is not transmitted from the portable terminal device 111 to the television 135. This is because transmission is prohibited when the voice command input button 121 is pressed. The same applies hereinafter.

  In step 46, the voice command registration processing unit 141C displays a voice command registration screen on the display unit 119, prompts the user to confirm the content of the key to be pressed, and uses the operation key 113 to follow the screen display and use the “record” key. Is pressed and, for example, “TV DENGEN” and a voice command 112 are generated within a certain time (for example, within 5 seconds) after the screen is switched toward the inner microphone 122. The post-recording voice command registration processing unit 141 </ b> C performs voice recognition processing on the voice command 112 with “TV dengen”. If you make a mistake, you can press the “Cancel” key to redo the pronunciation. The same applies to the explanation of pressing the “Cancel” key.

In step 47, the voice command registration processing unit 141C causes the display unit 119 to display the result recognized by the voice recognition processing. If there is no problem in the recognition result and the registration key is pressed by using the operation key 113, the voice command registration unit 126A of the memory 126 is caused to register the voice command 112 of “TV DENGEN”.
In step 48, when the registration process of the voice command 112 is completed, the voice command registration processing unit 141C causes the display unit 119 to display registration completion. By pressing the “OK” key, the function display screen becomes the standby screen of the mobile terminal device 111.

In this manner, the registration operation of the voice command 112 with “TV TV Dengen” is the registration of the TV remote control function calling operation and the power key pressing operation.
When the voice command 112 is registered for the key corresponding to the volume adjustment of the TV remote controller, the voice command 112 registration operation of “volume up” and “volume down” is similarly performed in steps 45 to 48. This registration operation registers the calling operation of the TV remote control function and the volume adjustment operation.

  When the voice command 112 is registered for the key corresponding to the channel switching of the TV remote controller, the registration operation of the voice command 112 of “channel up” and “channel down” is similarly performed in steps 45 to 48. This registration operation registers the calling operation of the TV remote control function and the channel switching operation.

FIG. 6 is a control flowchart for explaining the registration of the voice command 112 for the time signal in the first modification of FIG. As an example, a case where “Jihou” is registered in the voice command registration unit 126A as a voice command 112 for listening to “time signal” using the function of the telephone directory built in the portable terminal device 111 will be described with reference to FIG. .
In step 51, first, the function is displayed on the menu screen of the display unit 119 by using the operation key 113 by the user, thereby enabling selection of an icon of the telephone book.

  In step 52, when a phone book icon is selected, a phone book screen is displayed on the display unit 119, and a list of pairs of the name, name, and corresponding phone number is displayed on the phone book screen. Using the operation key 113, the “time signal 117” is selected on the display unit 119. In this embodiment, since it is desired to register the voice command 112 for making a call to the station number 117, the call key is pressed using the operation key 113 while pressing the voice command input button 121 in a state where the “time signal 117” is selected with the cursor. Press (off-hook key) to enter voice command registration mode. In this case, the call is not connected to the station number 117 by pressing the call key. This is because the voice command input button 121 is pressed.

  In step 53, when the call key is pressed while the voice command input button 121 is pressed with the "time signal 117" selected, the voice command registration processing unit 141C displays a voice command registration screen on the display unit 119. The user is prompted to confirm the contents of the key to be pressed, and according to the display on the screen, press the “record” key using the operation key 113 to face the inner microphone 122 within a certain time (for example, 5 Within a second), for example, the voice command 112 is pronounced “jiho”. The post-recording voice command registration processing unit 141C performs voice recognition processing on the voice command 112 for “Jiho”.

In step 54, the voice command registration processing unit 141C causes the display unit 119 to display the result recognized by the voice recognition processing. If there is no problem in the recognition result and the registration key is pressed using the operation key 113, the voice command registration unit 126A of the memory 126 is caused to register the voice command 112 “Jihou”.
In step 55, when the registration process of the voice command 112 is completed, the voice command registration processing unit 141C causes the display unit 119 to display registration completion. By pressing the “OK” key, the function display screen becomes the standby screen of the mobile terminal device 111.

  In this way, the registration operation of the voice command 112 with “Jihou” is the registration of the call operation of the telephone directory function and the operation of listening to the time signal.

FIG. 7 is a control flowchart for explaining the registration of the voice command 112 for the weather forecast in the second modification of FIG. As an example, a case where “tenkiyo” is registered in the voice command registration unit 126A as a voice command 112 for obtaining “weather forecast” information using the Internet function built in the portable terminal device 111 will be described with reference to FIG. explain.
In step 61, first, the function is displayed on the menu screen of the display unit 119 by using the operation key 113 by the user, and the Internet icon can be selected.

  In step 62, a text box for inputting a URL (Uniform Resource Locator) is displayed on the display unit 119 by selecting an Internet icon. The weather forecast URL is input in the text box, and the connection key is pressed according to the screen display. This makes it possible to browse your favorite weather forecast. In this embodiment, since the user wants to register a voice command 112 for browsing the weather forecast on the Internet, after inputting the URL of the user-specified weather forecast in the text box, press the voice command input button 121 and follow the on-screen connection key. Press to enter voice command registration mode. In this case, the connection key is not pressed to connect to the Internet. This is because the voice command input button 121 is pressed.

  In step 63, when the connection key is pressed while pressing the voice command input button 121, the voice command registration processing unit 141C displays a voice command registration screen on the display unit 119 and prompts the user to confirm the contents of the key to be pressed. In accordance with the display on the screen, the “record” key is pressed using the operation key 113, and within the predetermined time (for example, within 5 seconds) after the screen is switched toward the inner microphone 122, for example, “Tenkiyo” is sounded. The command 112 is sounded. The post-recording voice command registration processing unit 141C performs voice recognition processing on the voice command 112 for “Tenkiyoho”.

In step 64, the voice command registration processing unit 141C causes the display unit 119 to display the result recognized by the voice recognition processing. If there is no problem in the recognition result and the registration key is pressed using the operation key 113, the voice command registration unit 126A of the memory 126 is registered for the voice command 112 "Tenjoyo".
In step 65, when the registration process of the voice command 112 is completed, the voice command registration processing unit 141C causes the display unit 119 to display registration completion. By pressing the “OK” key, the function display screen becomes the standby screen of the mobile terminal device 111.

  In this way, the registration operation of the voice command 112 as “tenkiyo” is the registration of the Internet function call operation and the tenjojo browsing operation.

FIG. 8 is a control flowchart for explaining the registration of the voice command 112 to the camera shutter in the third modification of FIG. As an example, a case where “camera shutter” is registered in the voice command registration unit 126A as a voice command 112 for causing the camera 128 built in the portable terminal device 111 to perform a “camera shutter” operation will be described with reference to FIG. explain.
In step 71, first, the function display is performed on the menu screen of the display unit 119 by using the operation key 113 by the user, and the icon of the camera 128 can be selected.

  In step 72, when the icon of the camera 128 is selected, the screen of the camera 128 is displayed on the display unit 119, and a list of camera shutters, playback, and user settings is displayed on the screen of the camera 128. Using the operation key 113, the display unit 119 can select “camera shutter”. In this embodiment, since it is desired to register the voice command 112 for the character display corresponding to “camera shutter”, the character display corresponding to “camera shutter” is executed using the operation key 113 while pressing the voice command input button 121. Press according to the screen to enter the voice command registration mode. In this case, the camera shutter is not actually released. This is because the voice command input button 121 is pressed.

  If “camera shutter” is selected while pressing the voice command input button in step 73, the voice command registration processing unit 141C displays a voice command registration screen on the display unit 119 and confirms the contents of the key to be pressed. In accordance with the display on the screen, the “record” key is pressed using the operation key 113 and the screen is switched toward the inner microphone 122 within a predetermined time (for example, within 5 seconds), for example, “Camera” “Shutter” and voice command 112 are sounded. The post-recording voice command registration processing unit 141C performs voice recognition processing on the voice command 112 for “camera shutter”.

In step 74, the voice command registration processing unit 141C causes the display unit 119 to display the result recognized by the voice recognition processing. If there is no problem in the recognition result and the registration key is pressed by using the operation key 113, the voice command registration unit 126A of the memory 126 is caused to register the voice command 112 “camera shutter”.
In step 75, when the registration process of the voice command 112 is completed, the voice command registration processing unit 141C causes the display unit 119 to display registration completion. By pressing the “OK” key, the function display screen becomes the standby screen of the mobile terminal device 111.

In this manner, the registration operation of the voice command 112 for “camera shutter” is the registration of the function call operation of the camera 128 and the execution operation of the camera shutter.
In the case where the photographed picture is reproduced, when the voice command 112 is registered for the key corresponding to the reproduction of the camera 128, the registration operation of the voice command 112 as “Saisei” is similarly performed in steps 72 to 75. . In this registration operation, the camera function calling operation and the reproduction operation are registered.

  FIG. 9 is a control flowchart for explaining the registration of the voice command 112 to the server 136 by the radio wave 116 for the wireless LAN in the fourth modification of FIG. As an example, using this figure, using the access function to the server 136 by the radio wave 116 for the wireless LAN built in the portable terminal device 111, “printer output” is voiced as the voice command 112 for performing “printer output”. A case of registration in the command registration unit 126A will be described.

In step 81, first, when the user uses the operation key 113, a function is displayed on the menu screen of the display unit 119, and an icon for near field communication can be selected.
If the near field communication icon is selected in step 82, the near field communication menu screen is displayed on the display unit 119. The near field communication menu screen includes infrared communication, infrared remote control, BLUETOOTH, and wireless LAN. A list of functions is displayed in text. Using the operation key 113, the display unit 119 can select the character display of the wireless LAN.

  If the character display of the wireless LAN is selected in step 83, the wireless LAN menu screen is displayed on the display unit 119. The wireless LAN menu screen displays functions for printer output, file management, and user settings for the server 136. A list of characters is displayed. Using the operation key 113, it is possible to select the character display of the printer output on the display unit 119. In this embodiment, since it is desired to register the voice command 112 for the character display corresponding to “printer output”, the character display corresponding to “printer output” is executed using the operation key 113 while pressing the voice command input button 121. Press according to the screen to enter the voice command registration mode. In this case, there is no actual printer output. This is because the voice command input button 121 is pressed.

  In step 84, when the character display of the printer output is selected while pressing the voice command input button 121, the voice command registration processing unit 141C displays a voice command registration screen on the display unit 119, and displays the key to be pressed. The user confirms the content, presses the “record” key using the operation key 113 according to the display on the screen, and moves toward the inner microphone 122 within a certain time (for example, within 5 seconds) after the screen is switched. “Printer output” and a voice command 112 are generated. The post-recording voice command registration processing unit 141 </ b> C performs voice recognition processing of the voice command 112 with “printer output”.

In step 85, the voice command registration processing unit 141C causes the display unit 119 to display the result recognized by the voice recognition processing. If there is no problem in the recognition result and the registration key is pressed by using the operation key 113, the voice command registration unit 126A of the memory 126 is registered to process the voice command 112 “print printer”.
In step 86, when the registration process of the voice command 112 is completed, the voice command registration processing unit 141C causes the display unit 119 to display registration completion. By pressing the “OK” key, the function display screen becomes the standby screen of the mobile terminal device 111.

In this way, the registration operation of the voice command 112 with “printer output” is the registration of the operation of calling the wireless LAN function and the operation of performing printer output.
Further, when the voice command 112 is registered with respect to the key corresponding to the file management to the server 136, the voice command 112 is registered in the same manner in steps 83 to 86 as "file count". This registration operation is a registration of an access function call operation to the server 136 and a file management operation.

  FIG. 10 is a control flowchart for explaining the registration of the voice command 112 to the refrigerator 137 by the radio wave 117 for BLUETOOTH in the fifth modification of FIG. As an example, using this figure, “Zykokanri” is registered as a voice command 112 as a voice command 112 for “inventory management” using the remote control function to the refrigerator 137 by the radio wave 117 for BLUETOOTH built in the portable terminal device 111. A case of registration in the unit 126A will be described.

In step 91, first, when the user uses the operation key 113, a function is displayed on the menu screen of the display unit 119, and an icon for short-range communication can be selected.
In step 92, the near-field communication menu screen is displayed on the display unit 119 by selecting the near-field communication icon. The near-field communication menu screen includes functions of infrared communication, infrared remote control, BLUETOOTH, and wireless LAN. The list is displayed in text. Using the operation key 113, the display unit 119 enables selection of BLUETOOTH character display.

When the BLUETOOTH character display is selected in step 93, the BLUETOOTH menu screen is displayed on the display unit 119. The BLUETOOTH menu screen displays a list of refrigerators, home appliances including lighting, and user-set remote control functions. Characters are displayed.
If the refrigerator 137 is selected in step 94, the refrigerator menu screen is displayed on the display unit 119, and a list of functions for inventory management, freshness management, and user settings is displayed in text. Using the operation key 113, the display unit 119 can select the character display of the refrigerator. In this embodiment, since it is desired to register the voice command 112 for the character display corresponding to “inventory management”, the character display corresponding to “inventory management” is executed using the operation key 113 while pressing the voice command input button 121. Press according to the screen to enter the voice command registration mode. In this case, the BLUETOOTH radio wave is not actually output. This is because the voice command input button 121 is pressed.

  In step 95, when the character display of “inventory management” is selected while pressing the voice command input button 121, the voice command registration processing unit 141C displays the voice command registration screen on the display unit 119 and should be pressed. Prompt to confirm the contents of the key, and according to the display on the screen, press the “record” key using the operation key 113, toward the inner microphone 122 and within a certain time (for example, within 5 seconds) after the screen is switched. For example, the voice command 112 is pronounced “Zaikankanri”. The post-recording voice command registration processing unit 141 </ b> C performs voice recognition processing of the voice command 112 “Zaikokanri”.

In step 96, the voice command registration processing unit 141C causes the display unit 119 to display the result recognized by the voice recognition process. If there is no problem in the recognition result and the registration key is pressed by using the operation key 113, the voice command registration unit 126A of the memory 126 is caused to register the voice command 112 of “Zaikanri”.
In step 97, when the registration process of the voice command 112 is completed, the voice command registration processing unit 141C causes the display unit 119 to display registration completion. By pressing the “OK” key, the function display screen becomes the standby screen of the mobile terminal device 111.

In this way, the registration operation of the voice command 112 of “Zaikokanri” is the registration of the call operation of the remote control function of the refrigerator 137 and the operation of inventory management.
In this way, the registration operation of the voice command 112 of “Zaikokanri” is the registration of the operation of calling the function of the refrigerator 137 and the operation of inventory management.
Further, when the voice command 112 is registered for the key corresponding to the freshness management in the refrigerator 137, the voice command 112 is registered in the same manner in steps 94 to 97. This registration operation registers the call operation of the remote control function to the refrigerator 137 and the operation of freshness management.

  Further, when the voice command 112 is registered for a key corresponding to the lighting power source for a lighting (not shown) as a home appliance, similarly, in steps 93 to 97, the voice command 112 is registered as “tent” or “show”. Perform the action. In this registration operation, the calling operation of the remote control function to the illumination and the operation of the illumination power source are registered.

FIG. 11 is a control flowchart for explaining the registration of the voice command 112 for reading a mail in the sixth modification of FIG. As an example, a case where “mail reading” is registered in the voice command registration unit 126A as a voice command 112 for performing “mail reading” using the mail reading function will be described with reference to FIG.
In step 101, first, the function is displayed on the menu screen of the display unit 119 by using the operation key 113 by the user, and the mail icon can be selected.

  In step 102, when a mail icon is selected, a mail menu screen is displayed on the display unit 119, and a mail reading function is displayed in text on the mail menu screen. Using the operation key 113, it is possible to select a character display for reading a mail. In this embodiment, since it is desired to register the voice command 112 for the character display corresponding to “mail reading”, the character display corresponding to “mail reading” is executed using the operation key 113 while the voice command input button 121 is pressed. Press according to the screen to enter the voice command registration mode. In this case, mail reading is not actually executed. This is because the voice command input button 121 is pressed.

  In step 103, when the voice command input button 121 is pressed and the character display of "read aloud" is selected, the voice command registration processing unit 141C displays the voice command registration screen on the display unit 119 and should be pressed. Prompt to confirm the contents of the key, and according to the display on the screen, press the “record” key using the operation key 113, toward the inner microphone 122 and within a certain time (for example, within 5 seconds) after the screen is switched. For example, the voice command 112 is pronounced as “Mail Yomiage”. The post-recording voice command registration processing unit 141C performs voice recognition processing on the voice command 112 with “Mail Yomiage”.

In step 104, the voice command registration processing unit 141C causes the display unit 119 to display the result recognized by the voice recognition process. If there is no problem in the recognition result and the registration key is pressed by using the operation key 113, the voice command registration unit 126A of the memory 126 is registered to process the voice command 112 of “Mail Yomiage”.
In step 105, when the registration process of the voice command 112 is completed, the voice command registration processing unit 141C causes the display unit 119 to display registration completion. By pressing the “OK” key, the function display screen becomes the standby screen of the mobile terminal device 111.

In this manner, the registration operation of the voice command 112 with “Mail Yomiage” is the registration of the mail function calling operation and the mail reading operation.
Next, an operation example of the voice command processing unit 141B will be described. FIG. 12 is a control flow diagram illustrating an operation screen of the voice command processing unit 141B of the control unit 141 in FIG.

As shown in this figure, in steps 201A and 201B, first, the function display is performed on the menu screen of the display unit 119 by using the operation key 113 by the user, thereby enabling selection of an icon for voice input.
In step 202, the voice command processing mode is entered by selecting the voice input icon while pressing the voice command input button 121, and the voice command menu screen is displayed. On the voice command menu screen, the function name and the read operation name registered by the voice command registration processing unit 141C are displayed as follows, for example.

Function name Call name Camera and camera shutter
・ Seisei 2. TV remote control
·volume up
・ Volume down
・ Channel up
-Channel down 3. Reizoko ・ Zaikokanri Remote control ・ Sendkanri4. Internet ・ Tenkiyohou 5. Denwacho Jihou 6. Mail ・ Mail Yomiage7. Shomei Shomeidengen Remote control
8). Musenlan ・ Printer
・ File handling

In this way, a menu for displaying a list of all voice commands 112 registered in the voice command registration processing unit 141C of the user's portable terminal device 111 is provided, and the list can be managed. The voice command menu is selected, and the target voice command 112 can be selected and executed from the list display using the operation keys 113. Further, even if the user does not remember the read name that is the voice command 112, the read operation name can be pronounced if the display is viewed.
The voice command processing unit 141B recognizes the voice if the user pronounces TV dengen, compares the voice command 112 registered in the voice command registration unit 126A with the voice command 112, and matches the voice command 112 of the TV dengen. If accepted, the execution of the TV power ON / OFF operation is started.
In this way, it can be used like a shortcut function that can execute a function with few operations.

The list display menu has an editing function, and the array of the list display can be edited.
In step 203, for example, when the voice command processing unit 141B receives the voice command 112 with the television dengen, the television remote control is displayed on the display unit 119, and the operation of turning on and off the television dengen is displayed.

Further, the voice command processing unit 141B, for example, recognizes the voice if the user pronounces “mail yomiage”, compares it with the voice command 112 registered in the voice command registration unit 126A, and matches the voice command 112 of the mail yomiage. If it is received, the voice of the mail is synthesized and the mail is read out.
FIG. 13 is a flowchart illustrating an operation example of the voice command processing unit 141B of the control unit 141 in FIG.

  In step 201, first, the voice command processing unit 141B proceeds to step 202 when the user presses the voice command input button 121 in order to recognize the voice input from the microphone 122 or the microphone 124 as the voice command 112. If the user depresses the voice command input button 121 after releasing the voice command input button 121 to generate the voice command 112, the process proceeds to step 215.

In step 202, when the voice command input button 121 is pressed, the voice command processing unit 141B causes the speaker / microphone switching unit 141A to open or close the casing of the mobile terminal device 111 based on the detection information of the open / close detection sensor 127. To determine whether or not.
In step 203, the voice command processing unit 141B switches the microphone / speaker system switch 132, 133 to enable the inner microphone 122 and the inner speaker 123 when the casing of the mobile terminal device 111 is open, Proceed to 205.

In step 204, when the casing of the mobile terminal device 111 is closed, the voice command processing unit 141B switches the microphone / speaker system changeover switches 132 and 133 to enable the outer microphone 124 and the outer speaker 125.
In step 205, the voice command processing unit 141B performs an increment process of a voice command input counter (not shown).

In step 206, the voice command processing unit 141 </ b> B performs input processing of the voice command 112 from the inner microphone 122 or the outer microphone 124.
In step 207, the voice command processing unit 141B determines whether or not the upper limit value of the voice command input counter has been exceeded. If not exceeded, the process does not proceed to step 208, and the process is terminated, waits for exceeding, or waits for release of pressing of the voice command input button 121.

In step 208, when the upper limit value of the voice command input counter is exceeded, the voice command processing unit 141B performs voice recognition analysis processing on the input voice command 112.
In step 209, as a result of the voice recognition analysis processing, the voice command processing unit 141B has the voice command 112 registered in advance in the voice command registration unit 126A of the memory 126 as described with reference to FIGS. Determine whether they match. If not, the process proceeds to step 212.

In step 210, if they match, the voice command processing unit 141B outputs a command reception notification sound (beep) from the inner speaker 123 or the outer speaker 125 as a cue that the voice command 112 has been received and plays it.
In step 211, the voice command processing unit 141B corresponds to the voice command 112, executes a function and operation registered in advance, and ends this process.

In step 212, the voice command processing unit 141 </ b> B outputs a command non-acceptance notification sound (pipi) from the inner speaker 123 or the outer speaker 125 as a cue that the voice command 112 has not been accepted.
In step 213, the voice command processing unit 141B terminates the operation if there is a voice command process currently being operated.

In step 214, the voice command processing unit 141B sets the voice command input counter = 0, performs initialization, and ends this process.
In step 215, it is determined whether or not the voice command input counter = 0. If the voice command input counter = 0, the process ends. If the voice command input counter is not 0, the routine proceeds to step 208. When the user finishes inputting the voice command 112 and releases the press of the voice command input button 121, voice recognition analysis of the voice command 112 is performed.

Therefore, according to the present invention, the difficulty of multitasking operability by the small operation unit and display unit of the portable terminal device is improved. This is because the function execution by the voice command that does not require the screen display and the function execution by the key operation can be executed independently, so that the two functions can be easily used even in a small casing.
In addition, since voice commands can be registered for all functions / operations of the portable terminal device, even in a portable terminal device having multiple functions and a plurality of communication means, the means is taken care of for executing the desired function. Therefore, operability can be improved according to the user's preference.

In addition, since the functions of the user's portable terminal device can be executed / terminated only by a button operation and voice input, it can be easily operated even in the dark.
In addition, since the functions of the user's portable terminal device can be executed with the casing of the portable terminal device closed, the use of screen display, backlight, etc. can be reduced, and power can be saved.
Next, a representative example will be described with reference to the operation example of FIG.

With reference to the operation example of FIG. 13, the example of the function execution process by the audio | voice command 112 with respect to a time signal is demonstrated below.
For example, when the user listens to the time signal of the station number 117 while creating a mail using the operation key 113, the user keeps pressing the voice command input button 121 (step 201: Y) and the voice registered in advance within a predetermined time. The command 112 “Jihou” is pronounced toward the first microphone 122 and voice data is input (steps 202 to 206).

When the user releases the voice command input button 121 that has been pressed and pressed within the predetermined time (steps 201 and 215), the voice data input is subjected to voice command analysis processing by voice recognition (step 201). Step 208).
As a result of the analysis, if the voice command 112 registered in advance by the user matches, the station number registered by the user in advance as the corresponding operation of the voice command 112 “Jihou” together with the notification sound “pipi” (step 210). 117 dial execution is started (step 211).

When the telephone line during a call is disconnected after having finished listening to the time signal, the user can perform the end-of-call process by pressing only the voice command input button 121 for a short time with no speech.
When the voice command input button 121 is pressed for a short time without being silent, when the voice command input button 121 is released (step 201: N), it is determined that the voice command input counter is not 0 (step 215), and for voiceless input. Since the voice command analysis process (step 208: N) is performed, it is determined that the command is not accepted (step 209). Therefore, the command command notifying sound “pippi” (step 212) and the voice command 112 “ This is because “Jihou” is forcibly terminated (step 213). The same applies hereinafter.

  In this way, it is possible to listen to Jiho by voice input while creating a mail with the operation keys 113.

With reference to the operation example of FIG. 13, an example of a function execution process by the audio command 112 for the television 135 will be described below.
For example, when the user views the television 135 while creating a mail using the operation keys 113, the user keeps pressing the voice command input button 121 (step 201: Y) and the voice command registered in advance within a predetermined time. 112 “TV Dengen” is pronounced toward the inner microphone 122 of the first system, and voice data is input (steps 202 to 206).

When the user releases the voice command input button 121 which has been pressed and pressed within the predetermined time (steps 201 and 215), the voice data input is subjected to voice command analysis processing by voice recognition. (Step 208).
As a result of the analysis, when the voice command 112 registered in advance by the user is matched, the user has registered in advance as the corresponding operation of the voice command 112 “TV TV screen” together with the notification sound “pipi” (step 210). Execution of turning on or off the power of the television 135 is started (step 211).

  When the case of the portable terminal device 111 is closed after the mail is created, the second microphone 124 is switched to the outside microphone 124 and can be used as a remote control for volume adjustment of the television 135 and channel switching by voice input. Thus, since the remote control function is provided even when the casing of the portable terminal device 111 is closed, power consumption of the LCD backlight of the display unit 119 can be prevented.

  In this way, it is possible to perform remote control of the television 135 by voice input while creating a mail with the operation key 113.

  Referring to the operation example of FIG. 13, as a function execution process by the voice command 112 for the camera 128, the voice command input button 121 is pressed while creating a mail with the operation key 113, and the camera 128 is input by voice input of “camera shutter”. The photograph taken and the photograph taken can be pasted on the mail with the operation key 113.

  Referring to the operation example of FIG. 13, as an example of function execution processing by the voice command 112 for the server 136, the voice command input button 121 is pressed while the photograph of the camera 128 is played with the operation key 113, and “Printer Shrink” is performed. The necessary photos can be output to the server 136 by the voice input.

  Referring to the operation example of FIG. 13, as a function execution process by the voice command 112 for the refrigerator 137, the user presses the voice command input button 121 while using the operation key 113 and presses the voice command input button 121. You can get information on the type and amount of ingredients in the dishes and ask the other party to deliver or buy the missing ingredients.

Referring to the operation example of FIG. 13, with the casing of the portable terminal device 111 closed, the voice command input button 121 is pressed on the television 135 and the refrigerator 137 and voice input is performed using the outer microphone 124. The remote control function can be executed.
Normally, the remote control function of the mobile terminal device 111 is performed using the operation key 113 with the housing opened, but it is not necessary to use the operation key 113.

  In FIG. 3, the voice command processing unit 141B of the control unit 141 has been described as performing voice command analysis processing by voice recognition processing. However, the voice command processing unit 141C further includes a voiceprint analysis function, and the voice command registration processing unit 141C The voice print of the user's voice input from the inner microphone 122 or the outer microphone 124 is registered in the registration unit 126A, and the voice command processing unit 141B receives the voice input from the inner microphone 122 or the outer microphone 124 before the voice command voice recognition process. Analyzes the voiceprints. This makes it possible to prevent unauthorized use by a third party and malfunction due to the voice of a neighbor.

In FIG. 13, when the voice command 112 matches and the voice command 112 is accepted as a result of the voice command 112 analysis processing by the voice command processing unit 141B, the voice command acceptance notification sound “pipi” is sent to the inner speaker 123 or the outer speaker 125. However, the voice command processing unit 141B uses the inner speaker 123 or the voice command 112, for example, “Jihou” received by voice synthesis using voice data of 50 voices instead of the voice command notification sound “pipi”. It repeats from the outside speaker 125. Thereby, the user himself / herself can confirm whether or not the voice command 112 is correctly received.

  In FIG. 1, it has been described that the voice command input button 121 is pressed in order to recognize the voice input from the inner microphone 122 or the outer microphone 124 as the voice command 112. However, the voice command input button 121 needs to be a dedicated button. Alternatively, an existing operation key 113 including a numeric keypad and function keys, a pointing device, or the like may be substituted. As a result, the system can be simplified.

  The above description has been given with respect to a foldable portable terminal device, but the present invention is not limited to this, and can also be applied to a rod-shaped portable terminal device that is not foldable. It can also be used for a telephone) and a PDA (personal digital assistant).

BRIEF DESCRIPTION OF THE DRAWINGS It is a general view explaining the outline of the multitask system of the portable terminal device provided with the speech recognition function based on this invention. It is an external view which shows the outline of the portable terminal device 111 of FIG. It is a block diagram which shows schematic structure of the portable terminal device 111 of FIG. It is the schematic explaining the menu of the function display displayed on the display part 119. FIG. FIG. 4 is a control flow diagram illustrating registration of a voice command 112 for the remote control function of the television 135 using infrared rays 115 in the operation screen example of the voice command registration processing unit 141C of the control unit 141 in FIG. 3. It is a control flow figure explaining registration of voice command 112 to a time signal in the 1st modification of Drawing 5. FIG. 6 is a control flow diagram illustrating registration of a voice command 112 for a weather forecast in the second modification of FIG. 5. It is a control flowchart explaining registration of the audio | voice command 112 with respect to a camera shutter in the 3rd modification of FIG. FIG. 10 is a control flow diagram for explaining registration of a voice command 112 to the server 136 by the radio wave for wireless LAN 116 in the fourth modification of FIG. 5. It is a control flow figure explaining registration of the voice command 112 with respect to the refrigerator 137 by the radio wave 117 for BLUETOOTH in the 5th modification of FIG. It is a control flowchart explaining registration of the voice command 112 with respect to mail reading in the 6th modification of FIG. FIG. 4 is a control flow diagram illustrating an operation screen of a voice command processing unit 141B of the control unit 141 in FIG. 3. It is a figure which shows the flowchart explaining the operation example of the voice command processing part 141B of the control part 141 in FIG.

Explanation of symbols

111 ... mobile terminal device 112 ... voice command 113 ... operation key 114 ... radio wave 115 for base station ... infrared ray 116 ... radio wave 117 for wireless LAN ... radio wave 118 for BLUETOOTH ... antenna 119 ... display unit 121 ... voice command input button 122 ... Inner microphone 123 ... inner speaker 124 ... outer microphone 125 ... outer speaker 126 ... memory 126A ... voice command registration unit 127 ... open / close detection sensor 128 ... camera 129 ... Ir input / output circuit 130 ... BLUETOOTH circuit 131 ... microphone / speaker system switching signal 132 133 ... Microphone / speaker system switch 134 ... Telephone line / Internet 135 ... Television 136 ... Server 137 ... Refrigerator 140 ... Wireless unit 141 ... Control unit 141A ... Speaker / microphone switching unit 1 1B ... voice command processing unit 141C ... voice command registration processing unit

Claims (19)

In a multitask system of a mobile terminal device having a voice recognition function,
A voice command registration unit for registering a voice command that defines function call and function execution of the mobile terminal device;
A voice command input button for inputting voice from the microphone of the portable terminal device as a voice command during the pressing period;
A voice command registration processing unit that performs voice recognition processing on voice input to the microphone during a period during which the voice command input button is pressed, and registers a recognition result in the voice command registration unit as a voice command;
A voice command is input from the microphone during the pressing period of the voice command input button, the voice command input after releasing the pressing of the voice command input button is subjected to voice recognition processing, and a recognition result is registered in the voice command registration unit. A mobile terminal having a voice recognition function, comprising: a voice command processing unit that calls a function corresponding to the received voice command and executes a function operation when the voice command is received in accordance with the received voice command Multitask system of equipment. The mobile phone with a voice recognition function according to claim 1, wherein the operation key of the mobile terminal device and the voice command processing unit independently call a function of the mobile terminal device and execute the function. A multitasking system for terminal equipment. The voice command registration processing unit displays a plurality of functions of the mobile terminal device as icons on the display unit, and the voice input to the microphone during the pressing period of the voice command input button for the selected function. The multitask system for a portable terminal device having a voice recognition function according to claim 1, wherein the recognition result is registered as a voice command. The voice command registration processing unit displays a plurality of functions of the mobile terminal device as icons on the display unit, displays a selected function name, and guides recording of pronunciation of the function name, voice recognition after recording The multitask system for a portable terminal device with a voice recognition function according to claim 1, wherein a screen for guiding registration of the processed recognition result and a screen for completion of registration are displayed. When the voice command input button is pressed for the voice input function selected from the icon display displayed on the display unit so that a plurality of functions of the portable terminal device can be selected, The mobile terminal device with voice recognition function according to claim 1, wherein a list of voice commands registered in the voice command registration unit is displayed, and a target voice command can be selected and executed. Task system. The voice recognition according to claim 1, wherein the voice command processing unit sounds a command acceptance notification sound when receiving a voice command as a recognition result, and sounds a command non-acceptance notification sound when not receiving a voice command. A multitasking system for mobile terminals with functions. The voice recognition function according to claim 1, wherein the voice command processing unit has a voice synthesis function, and when the voice command is received, the voice command processing unit repeats the received voice command from the speaker of the portable terminal device by voice synthesis. A multitasking system for portable terminal devices equipped with The voice command processing unit inputs a voice command from the microphone during the pressing period of the voice command input button, and if the predetermined time has elapsed before the voice command input button is released, The multitask system for a portable terminal device having a voice recognition function according to claim 1, wherein recognition processing is performed. The voice command processing unit forcibly terminates the operation of the function that is being executed when the voice command input button is silently pressed after the function operation is performed and the voice command is not input and the press is released. The multitask system of the portable terminal device provided with the speech recognition function of Claim 1 characterized by these. The voice command processing unit has a voiceprint analysis function, the voice command registration processing unit C causes the voice command registration unit to register a user's voiceprint input from a microphone, and the voice command processing unit The multitask system for a portable terminal device having a voice recognition function according to claim 1, wherein voiceprints input from a microphone are analyzed before the recognition process. The multi-task system of a portable terminal device having a voice recognition function according to claim 1, wherein the voice command input button is a non-lock type button and is located on a side surface of the portable terminal device. 2. The portable terminal device having a voice recognition function according to claim 1, wherein the voice command input button is used by selecting one of a numeric keypad, a function key, and a pointing device of the portable terminal device. Multitasking system. 2. The voice command processing according to claim 1, wherein the voice command processing performs a call of a function of a mail, a camera, a calculator, a memo, the Internet, a dictionary, a telephone directory, a schedule, a remote controller, a voice input function, and a function operation. Multitask system for mobile terminal devices with voice recognition function. The function of the remote control includes a remote control function of a home appliance including a television using infrared rays, a server using radio waves for wireless LAN, a refrigerator using radio waves for BLUETOOTH, and lighting. A multitask system of a portable terminal device having the described voice recognition function. While the multi-task system of the portable terminal device reads out the mail function with the operation key and creates the mail, the voice command processing unit calls the function of the phone book and enables the operation processing of the function of listening to the time signal. A multitask system for a portable terminal device having the voice recognition function according to claim 1. The portable terminal device is a foldable type, and has a pair of microphones and speakers on the inside and outside, and an open / close detection sensor for detecting opening and closing of the portable terminal device, and when the portable terminal device is open, When only the pair of microphones and speakers is enabled and the portable terminal device is closed, only the pair of outer microphones and speakers are enabled, and the voice command processing unit is enabled when the portable terminal device is open. 2. The mobile phone with a voice recognition function according to claim 1, wherein a voice command is input from an inner microphone, and a voice command is input from an outer microphone when the portable terminal device is closed. A multitasking system for terminal equipment. In the multi-task system of the portable terminal device, the voice command processing unit calls the television function and turns on the television power supply while the portable terminal device is opened and the mail function is read with an operation key to create a mail. The operation of the function to insert is enabled, and the operation of the function of adjusting the volume of the television and switching the channel is performed in a state in which the creation of the mail by the operation key is completed and the portable terminal device is closed. A multitask system of a mobile terminal device having the described voice recognition function. In the multi-task system of the portable terminal device, the voice command processing unit calls a television function in a state where the portable terminal device is closed, and operates a function for turning on a television, adjusting a volume, and switching a channel. The multitask system for a portable terminal device having a voice recognition function according to claim 16, wherein operation is performed. In a multitasking method of a mobile terminal device having a voice recognition function,
Selecting one of a plurality of functions of the mobile terminal device and inputting a voice from the microphone of the mobile terminal device during a period of pressing a voice command input button;
Voice recognition processing of the input voice;
Registering the recognition result of the speech recognition processing as a voice command that defines the function call of the mobile terminal device and the execution of the function;
Inputting a voice command to the microphone during a period of pressing of the voice command input button;
Performing voice recognition processing of a voice command input after releasing the voice command input button;
Accepting a voice command when the recognition result obtained by performing voice recognition processing on the voice command matches the registered voice command;
A multitasking method for a mobile terminal device having a voice recognition function, comprising: calling a function of the mobile terminal device corresponding to the received voice command and executing a function operation.

Images