JP2014225796A - Electronic apparatus, control method and control program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To improve a processing executed via an interactive interface.SOLUTION: The electronic apparatus (for example, smartphone 1) includes: a mic; and a controller that performs a voice control based on the voice input through the mic. The controller terminates a voice input of a character string to be inserted in an input field of an application selected by the voice control through the voice control. That is, the electronic apparatus (for example, smartphone 1) starts and terminates to input of a large text in a predetermined input field of an application under a voice control via an interactive interface.

Description

  The present application relates to an electronic device, a control method, and a control program.

  Touch screen devices comprising a touch screen are known. Examples of touch screen devices include, but are not limited to, electronic devices such as mobile phones, smartphones, and tablets. The touch screen device detects a finger, pen, or stylus pen gesture via the touch screen. The touch screen device operates according to the detected gesture. An example of the operation according to the detected gesture is described in Patent Document 1, for example.

  The basic operation of the touch screen device is realized by an OS (Operating System) installed in the device. Examples of the OS installed in the touch screen device include, but are not limited to, Android (registered trademark), BlackBerry (registered trademark) OS, iOS, Symbian (registered trademark) OS, and Windows (registered trademark) Phone.

  In recent years, among the above-mentioned electronic devices, those equipped with an interactive interface that enables operations not only by gestures but also by voice input have begun to become widespread. These electronic devices realize operations such as application activation and Internet search, for example, by voice.

International Publication No. 2008/086302

  An electronic device having an interactive interface has room for improvement in processing via the interactive interface.

  An electronic device according to one aspect includes a microphone and a controller that performs voice control based on voice input through the microphone, and the controller is provided in an input field of an application selected by the voice control. The voice input of the character string to be inserted is terminated by the voice control.

  The control method which concerns on one aspect is a control method which controls the electronic device which has a microphone, Comprising: The step which performs the audio | voice control based on the audio | voice input via the said microphone, The application selected by the said audio | voice control Ending voice input of a character string to be inserted into the input field by the voice control.

  A control program according to one aspect includes a step of executing voice control based on voice input through the microphone on an electronic device having a microphone, and a character to be inserted into an input field of an application selected by the voice control A step of ending the voice input of the row by the voice control.

FIG. 1 is a block diagram of a smartphone according to the embodiment. FIG. 2 is a diagram illustrating an example of processing in the case where voice input to the body field of the mail application is terminated by detection of a predetermined keyword. FIG. 3 is a diagram illustrating an example of processing in the case where voice input to the body field of the mail application is terminated by detection of a predetermined keyword. FIG. 4 is a diagram illustrating another example of processing when the voice input to the body field of the mail application is terminated by detecting a predetermined keyword. FIG. 5 is a diagram illustrating another example of processing in the case where voice input to the body field of the mail application is terminated by detection of a predetermined keyword. FIG. 6 is a diagram illustrating an example of processing in the case where the voice input to the body field of the mail application is terminated when the number of voice inputs is equal to or greater than the threshold value. FIG. 7 is a diagram illustrating an example of processing in the case where the voice input to the body field of the mail application is terminated when the number of voice inputs is equal to or greater than the threshold. FIG. 8 is a flowchart illustrating an example of a processing procedure performed by the smartphone according to the embodiment. FIG. 9 is a flowchart illustrating an example of a processing procedure by the smartphone according to the embodiment.

  Embodiments for carrying out an electronic device, a control method, and a control program according to the present application will be described in detail with reference to the drawings. Hereinafter, a smartphone will be described as an example of an electronic device.

(Embodiment)
A functional configuration of the smartphone 1 according to the embodiment will be described with reference to FIG. FIG. 1 is a block diagram of a smartphone according to the embodiment. In the following description, the same code | symbol may be attached | subjected to the same component. A duplicate description may be omitted.

  As shown in FIG. 1, the smartphone 1 includes a touch screen display 2, a button 3, an illuminance sensor 4, a proximity sensor 5, a communication unit 6, a receiver 7, a microphone 8, a storage 9, and a controller 10. A speaker 11, a camera 12, a posture detection unit 15, and a vibrator 18.

  The touch screen display 2 includes a display 2A and a touch screen 2B. The display 2 </ b> A and the touch screen 2 </ b> B may be arranged, for example, may be arranged side by side, or may be arranged apart from each other. When the display 2A and the touch screen 2B are arranged so as to overlap each other, for example, one or more sides of the display 2A may not be along any side of the touch screen 2B.

  The display 2A includes a liquid crystal display (LCD: Liquid Crystal Display), an organic EL display (OELD: Organic Electro-Luminescence Display), or an inorganic EL display (IELD: Inorganic Electro-Luminescence Display). The display 2A displays characters, images, symbols, graphics, and the like. Screens including characters, images, symbols, graphics, and the like displayed on the display 2A include a screen called a lock screen and a screen called a home screen. The home screen is sometimes called a desktop, a standby screen, an idle screen, or a standard screen.

  The touch screen 2B detects contact of a finger, a pen, a stylus pen, or the like with respect to the touch screen 2B. The touch screen 2B can detect a position where a plurality of fingers, a pen, a stylus pen or the like (hereinafter simply referred to as “finger”) contacts the touch screen 2B. The touch screen 2B notifies the controller 10 of the contact of the finger with the touch screen 2B (touch screen display 2) together with the position on the touch screen display 2 of the touched location.

  The detection method of the touch screen 2B may be any method such as a capacitance method, a resistive film method, a surface acoustic wave method (or an ultrasonic method), an infrared method, an electromagnetic induction method, and a load detection method. In the following description, in order to simplify the description, it is assumed that the user uses the finger to touch the touch screen 2B in order to operate the smartphone 1.

  The controller 10 (smart phone 1) has at least one of the contact detected by the touch screen 2B, the position at which the contact is detected, the change in the position at which the contact is detected, the interval at which the contact is detected, and the number of times the contact is detected. The type of gesture is determined based on the one. The gesture is an operation performed on the touch screen 2B (touch screen display 2) using a finger. The gestures that the controller 10 (smart phone 1) determines via the touch screen 2B include, for example, touch, long touch, release, swipe, tap, double tap, long tap, drag, flick, pinch in, and pinch out. However, it is not limited to these.

  The button 3 receives an operation input from the user. The number of buttons 3 may be singular or plural.

  The illuminance sensor 4 detects the illuminance of the ambient light of the smartphone 1. Illuminance indicates light intensity, brightness, or luminance. The illuminance sensor 4 is used for adjusting the luminance of the display 2, for example.

  The proximity sensor 5 detects the presence of a nearby object without contact. The proximity sensor 5 detects the presence of an object based on a change in a magnetic field or a change in a feedback time of an ultrasonic reflected wave. For example, the proximity sensor 5 detects that the display 2 is brought close to the face. The illuminance sensor 4 and the proximity sensor 5 may be configured as one sensor. The illuminance sensor 4 may be used as a proximity sensor.

  The communication unit 6 communicates wirelessly. The wireless communication standards supported by the communication unit 6 include, for example, cellular phone communication standards such as 2G, 3G, and 4G, and short-range wireless communication standards. Cellular phone communication standards include, for example, LTE (Long Term Evolution), W-CDMA (Wideband Code Division Multiple Access), WiMAX (Worldwide PDA), WiMAX (Worldwide Interoperability Pc) ) (Global System for Mobile Communications), PHS (Personal Handy-phone System), and the like. Examples of short-range wireless communication standards include IEEE 802.11, Bluetooth (registered trademark), IrDA (Infrared Data Association), NFC (Near Field Communication), and WPAN (Wireless Personal Area Network). As a communication standard of WPAN, for example, there is ZigBee (registered trademark). The communication unit 6 may support one or more of the communication standards described above.

  The communication unit 6 receives a radio signal in a predetermined frequency band from a GPS satellite, demodulates the received radio signal, and sends the processed signal to the controller 10. The smartphone 1 may disperse the communication function with the GPS satellite from the communication unit 6 and provide an individual communication unit independent from the communication unit 6.

  The receiver 7 is a sound output unit. The receiver 7 outputs the sound signal transmitted from the controller 10 as sound. The receiver 7 is used, for example, to output the other party's voice during a call. The microphone 8 is a sound input unit. The microphone 8 converts the user's voice or the like into a sound signal and transmits the sound signal to the controller 10.

  The storage 9 stores programs and data. The storage 9 is also used as a work area for temporarily storing the processing result of the controller 10. The storage 9 may include any non-transitory storage medium such as a semiconductor storage medium and a magnetic storage medium. The storage 9 may include a plurality of types of storage media. The storage 9 may include a combination of a portable storage medium such as a memory card, an optical disk, or a magneto-optical disk and a storage medium reader. The storage 9 may include a storage device used as a temporary storage area such as a RAM (Random Access Memory).

  The programs stored in the storage 9 include an application executed in the foreground or the background and a control program that supports the operation of the application. For example, an application executed in the foreground displays a screen on the display 2. The control program includes an OS, for example. The application and the control program may be installed in the storage 9 via wireless communication by the communication unit 6 or a non-transitory storage medium.

  The storage 9 stores, for example, a control program 9A, a call application 9B, an SMS application 9C, an email application 9D, a memo pad application 9E, a moving image application 9F, a browser application 9G, telephone book data 9Y, setting data 9Z, and the like.

  The control program 9A provides functions related to various controls for operating the smartphone 1. The control program 9A provides an interactive interface function for executing voice control based on voice input via the microphone 8. The interactive interface function includes a function for causing the display 2A to display an execution screen indicating that the interactive interface function is being executed. The interactive interface function includes a function of recognizing a voice input by the microphone 8 and responding with a voice and characters according to the recognition result.

  The control program 9A provides a function of recognizing voice input by the user and executing processing based on the recognition result. This function includes a function of automatically executing an application corresponding to the recognition result from the applications stored in the storage 9 when the speech recognition is successful.

  The control program 9A provides a function of recognizing a voice input by the user and displaying a list of applications stored in the storage 9 when the process cannot be executed based on the recognition result. When the process cannot be executed based on the recognition result, for example, when the voice recognition fails or when it is determined that the application corresponding to the recognition result cannot be detected from the applications stored in the storage 9 Is included.

  The control program 9A provides a function for ending voice input of a character string to be inserted into an input field of an application selected by voice control via an interactive interface function by voice control. Specifically, when the control program 9A detects a predetermined keyword input by voice through the interactive interface or determines that the number of input voices through the interactive interface is greater than or equal to a threshold value, Provided is a function for terminating voice input of a character string to be inserted into an input field of an application selected by voice control.

  For example, when the email application 9D is selected through the interactive interface, the control program 9A causes the controller 10 to start voice control related to the email application 9D. In the present embodiment, it is assumed that the e-mail application 9D and the memo pad application 9E are registered in advance as applications that should execute voice control. In the present embodiment, the voice control related to voice control by the email application 9D means that voice input can be performed multiple times as a voice corresponding to a character string to be inserted into the body field of the email application 9D. This corresponds to the control for terminating the voice input according to the detection of a predetermined keyword or the number of voice inputs. Subsequently, the control program 9A stores a character string inputted by voice as a character string to be inserted into the body field of the email application 9D until the predetermined keyword is detected in the email application 9D. The controller 10 is caused to realize processing to be stored in the storage area. When a predetermined keyword is input, the control program 9A causes the controller 10 to realize a process of ending the voice input of the character string to be inserted into the body field of the email application 9D. Alternatively, the control program 9A starts the input of the voice corresponding to the character string to be inserted into the body field of the email application 9D until the number of voice inputs reaches or exceeds the threshold value. As a character string to be inserted into the text field, the controller 10 is caused to realize a process of storing a character string input by voice in a storage area of the storage 9. The control program 9A causes the controller 10 to realize a process of ending the voice input of the character string to be inserted into the body field of the email application 9D when the number of voice inputs exceeds the threshold.

  The control program 9A provides a function for starting an application selected by voice control when voice input is finished by voice control and inserting a character string input by voice into the input field of the started application.

  For example, when the e-mail application 9D is selected through the interactive interface, the control program 9A activates the e-mail application 9D when the voice input is terminated by the above-described voice control, and the text of the activated e-mail application 9D. The character string stored in the storage 9 is inserted as a character string to be inserted into the body field of the email application 9D.

  In addition, the control program 9A controls the communication unit 6 and the like, thereby enabling LTE (Long Term Evolution), W-CDMA (Wideband Code Division Multiple Access), WiMAX (Worldwide InterCirability). A function for realizing communication using Cellular, GSM (registered trademark) (Global System for Mobile Communications), PHS (Personal Handy-phone System), and the like is provided.

  The control program 9A controls the communication unit 6 and the like, thereby using IEEE 802.11, Bluetooth (registered trademark), IrDA (Infrared Data Association), NFC (Near Field Communication), WPAN (Wireless Personal Network), and the like. A function for realizing near field communication is provided.

  The control program 9A provides a function for realizing a call by controlling the communication unit 6 and the microphone 8, for example.

  The functions provided by the control program 9A may be used in combination with functions provided by other programs stored in the storage 9, such as the call application 9B or the e-mail application 9D, when realizing a call. For example, the interactive interface function provided by the control program 9A and other functions provided by the control program 9A may be used in combination. Each function provided by the control program 9A may be divided into a plurality of program modules, or may be combined with other programs. For example, the interactive interface function provided by the control program 9A may be divided into other modules.

  The call application 9B provides a function for realizing a call. The SMS application 9C provides a function for sending and receiving mail within a predetermined number of characters as SMS (Short Message Service). The email application 9D provides an email function for creating, sending, receiving, and displaying an email. The memo pad application 9E provides a text editor function for creating and managing text data. The moving image application 9F provides a function for reproducing a moving image. The browser application 9G provides a browsing function. The application stored in the storage 9 is only an example. In addition to the one shown in FIG. 1, a plurality of applications such as a navigation application, a music player application, a computer application, and a tag application may be stored.

  The phone book data 9Y includes information such as a telephone number and a mail address. The setting data 9Z includes information regarding various settings and processes related to the operation of the smartphone 1. For example, the setting data 9Z includes threshold information for ending the voice control. In addition, the storage 9 stores character string data input by voice through the microphone 8 by processing by the controller 10.

  The controller 10 is an arithmetic processing device. The arithmetic processing unit includes, for example, a CPU (Central Processing Unit), an SoC (System-on-a-chip), an MCU (Micro Control Unit), and an FPGA (Field-Programmable Gate Array), but is not limited thereto. The controller 10 controls various operations of the smartphone 1 to realize various functions.

  Specifically, the controller 10 executes instructions included in the program stored in the storage 9 while referring to the data stored in the storage 9 as necessary. The controller 10 controls the functional unit according to data and instructions, thereby realizing various functions. The functional unit includes, for example, the display 2, the communication unit 6, the receiver 7, the microphone 8, and the speaker 11, but is not limited thereto. The controller 10 may change the control according to the detection result of the detection unit. The detection unit includes, for example, the button 3, the illuminance sensor 4, the proximity sensor 5, the microphone 8, the camera 12, the posture detection unit 15, and the touch screen 21, but is not limited thereto.

  The controller 10 realizes processing by an interactive interface by executing the control program 9A. For example, the controller 10 causes the display 2A to display an execution screen indicating that the interactive interface function is being executed. For example, the controller 10 recognizes the voice input by the microphone 8 and responds with voice and characters according to the recognition result.

  The controller 10 recognizes the voice input by the user by executing the control program 9A, and realizes processing based on the recognition result. For example, when the recognition of the voice is successful, the controller 10 automatically activates an application corresponding to the recognition result from among the applications stored in the storage 9.

  The controller 10 recognizes the voice input by the user by executing the control program 9A, and displays a list of applications stored in the storage 9 when the process cannot be executed based on the recognition result. Realize. When the process cannot be executed based on the recognition result, for example, when the voice recognition fails or when it is determined that the application corresponding to the recognition result cannot be detected from the applications stored in the storage 9 Is included.

  By executing the control program 9A, the controller 10 realizes a process for terminating the voice input of the character string to be inserted into the input field of the application selected by the voice control through the interactive interface function by the voice control. . Specifically, when the controller 10 detects a predetermined keyword input by voice through the interactive interface or determines that the number of input voices through the interactive interface is equal to or greater than a threshold, The voice input of the character string to be inserted into the input field of the application selected by the control is terminated.

  For example, when the e-mail application 9D is selected via the interactive interface, the e-mail application 9D starts voice control related to voice control. In the present embodiment, it is assumed that the e-mail application 9D and the memo pad application 9E are registered in advance as applications that should execute voice control. In the present embodiment, for example, the voice control related to voice control by the email application 9D means that voice input can be performed multiple times as a voice corresponding to a character string to be inserted into the body field of the email application 9D. This corresponds to the control for terminating the voice input according to the detection of a predetermined keyword or the number of voice inputs. Subsequently, until the predetermined keyword is detected, the controller 10 saves the character string to be input as a character string to be inserted into the body field of the email application 9D in the storage area of the storage 9. When a predetermined keyword is input, the controller 10 ends the voice input of the character string to be inserted into the body field of the email application 9D. Alternatively, the controller 10 starts the text input corresponding to the character string to be inserted into the text field of the email application 9D until the number of voice input exceeds the threshold, and then the text of the email application 9D. As a character string to be inserted into this field, a character string inputted by voice is stored in the storage area of the storage 9. When the number of input voices exceeds the threshold, the controller 10 ends the voice input of the character string to be inserted into the body field of the email application 9D.

  When the controller 10 executes the control program 9A and ends the voice input by voice control, the controller 10 starts the application selected by voice control and inserts the character string input by voice into the input field of the started application. Provides the functionality of

  For example, when the e-mail application 9D is selected via the interactive interface, the controller 10 activates the e-mail application 9D when the voice input is ended by the above-described voice control, and the body of the activated e-mail application 9D is displayed. The character string stored in the storage 9 is inserted as the character string to be inserted into the body field of the email application 9D.

  The speaker 11 is a sound output unit. The speaker 11 outputs the sound signal transmitted from the controller 10 as sound. The speaker 11 is used for outputting a ring tone and music, for example. One of the receiver 7 and the speaker 11 may also function as the other.

  The camera 12 converts the captured image into an electrical signal. The camera 12 includes, for example, an in-camera that captures an object facing the display 2 and an out-camera that captures an object facing the opposite surface of the display 2.

  The posture detection unit 15 detects the posture of the smartphone 1. The posture detection unit 15 includes at least one of an acceleration sensor, a direction sensor, and a gyroscope in order to detect the posture.

  The vibrator 18 vibrates a part or the whole of the smartphone 1. The vibrator 18 includes, for example, a piezoelectric element or an eccentric motor in order to generate vibration. The vibration by the vibrator 18 is used to notify the user of various events such as incoming calls.

  The functional configuration of the smartphone 1 illustrated in FIG. 1 is an example, and may be appropriately changed within a range that does not impair the gist of the present invention.

  An example of processing executed by the smartphone 1 according to the embodiment will be described with reference to FIGS. F1 appearing in FIGS. 2 to 7 indicates a user's finger. 2 to 7, an example of processing when a so-called lock screen is displayed on the display 2 </ b> A will be described, but similar processing can be realized as long as an interactive interface can be activated. In the following description, what is described as “operation” may be any touch gesture that can be detected by the smartphone 1 (controller 10) via the touch screen 2B.

  2 and 3 are diagrams illustrating an example of processing when voice input to a body field of a mail application (for example, the email application 9D) is terminated by detection of a predetermined keyword.

  As shown in FIG. 2, the smartphone 1 displays a lock screen 40 on the display 2A. The lock screen 40 is displayed when the smartphone 1 is in a locked state. When the lock screen 40 is displayed on the display 2 </ b> A, the smartphone 1 does not accept a user operation except for a specific operation. The specific operation includes, for example, an operation for releasing the lock state and an operation for displaying a notification screen on the display 2A. The lock state may include not only a state in which a user operation is not accepted but also a security lock state. As shown in FIG. 2, the lock screen 40 is provided with an icon 40a for starting an interactive interface.

  As shown in FIG. 2, when the smartphone 1 detects an operation on the icon 40a disposed on the lock screen 40 via the touch screen 2B (step S11), the smartphone 1 displays an interactive interface execution screen 50 on the display 2A. Display (step S12). At this time, as shown in step S12, the smartphone 1 outputs the voice guidance “Please tell me your business.” From the speaker 11 and displays the speech balloon image (S-1) corresponding to the voice guidance on the execution screen. 50.

  Subsequently, when a voice is input from the microphone 8, the smartphone 1 executes a process for recognizing the input voice. When the recognition is successful, the smartphone 1 determines whether there is an application corresponding to the recognition result. When there is an application corresponding to the recognition result, the smartphone 1 determines whether to perform voice control related to the application. When the smartphone 1 executes voice control related to the application, the smartphone 1 responds to the user and starts voice control related to the application (steps S13 to S14).

  Specifically, when the smartphone 1 recognizes that the character string input by voice through the microphone 8 is “mail to Mr. Ichiro Patent”, a speech bubble corresponding to the recognized character string as shown in step S13. The image (U-1) is displayed on the execution screen 50. Subsequently, the smartphone 1 detects an e-mail application 9D corresponding to the recognition result from the applications stored in the storage 9, and determines that voice control related to the e-mail application 9D is executed. Subsequently, as shown in step S <b> 14, the smartphone 1 starts voice control of the email application 9 </ b> D, and displays an image 50 a for notifying the user on the execution screen 50. As shown in step S <b> 14, the smartphone 1 outputs a voice guidance “Please tell me the subject” from the speaker 11 and displays a speech balloon image (S-2) corresponding to the voice guidance on the execution screen 50. .

  Subsequently, when a voice is input from the microphone 8, the smartphone 1 executes a process for recognizing the input voice. When the recognition is successful, the smartphone 1 stores the recognition result and responds to the user (Steps S15 to S16).

  Specifically, when the smartphone 1 recognizes that the character string input by voice through the microphone 8 is “Good morning”, a speech balloon corresponding to the recognized character string is displayed as shown in step S15. The image (U-2) is displayed on the execution screen 50. Subsequently, the smartphone 1 stores the recognized character string, and displays the image 50b in which the recognized character string is inserted in the field corresponding to the subject of the image 50a in which the subject has not been input (step S16). After obtaining the subject of the mail, as shown in step S16, the smartphone 1 outputs a voice guidance from the speaker 11 “Please tell me the text. A corresponding balloon image (S-3) is displayed on the execution screen 50.

  Subsequently, when a voice is input from the microphone 8, the smartphone 1 executes a process for recognizing the input voice. If the recognition is successful, the smartphone 1 saves the recognition result and responds to the user on the condition that the recognition result is not the end keyword of the voice input or that the number of times of voice input has not reached the threshold. (Step S17 to Step S18).

  Specifically, when the smartphone 1 recognizes that the character string input by voice through the microphone 8 is “8 o'clock set”, a speech bubble corresponding to the recognized character string as shown in step S17. The image (U-3) is displayed on the execution screen 50. Subsequently, the smartphone 1 stores the recognized character string, and displays the image 50c in which the recognized character string is inserted in the field corresponding to the text of the image 50b in which the text has not been input (step S18). After obtaining the character string to be inserted into the body of the mail, as shown in step S18, the smartphone 1 again receives a voice guidance from the speaker 11 “Tell me the body. While outputting, the balloon image (S-4) corresponding to the voice guidance is displayed on the execution screen 50.

  Subsequently, when a voice is input from the microphone 8, the smartphone 1 executes a process for recognizing the input voice. If the recognition is successful, the smartphone 1 saves the recognition result and responds to the user on the condition that the recognition result is not the end keyword of the voice input or that the number of times of voice input has not reached the threshold. (Step S19 to Step S20).

  Specifically, when the smartphone 1 recognizes that the character string input by voice through the microphone 8 is “Everyone gathered”, as shown in step S19, the smartphone 1 corresponds to the recognized character string. The balloon image (U-4) is displayed on the execution screen 50. Subsequently, the smartphone 1 stores the recognized character string, and displays the image 50d in which the recognized character string is inserted in the field corresponding to the text of the image 50c (step S20). After re-acquiring the character string to be inserted into the body of the mail, as shown in step S20, the smartphone 1 again receives a voice guidance “Please tell me the body. And an image of a balloon (S-5) corresponding to the voice guidance is displayed on the execution screen 50.

  Subsequently, when a voice is input from the microphone 8, the smartphone 1 executes a process for recognizing the input voice. If the recognition is successful, the smartphone 1 ends the voice input and starts the application on the condition that the recognition result is a voice input end keyword or that the number of voice inputs is equal to or greater than a threshold (step S21). -Step S22).

  Specifically, when the smartphone 1 recognizes that the character string input by voice through the microphone 8 is “complete”, which is a keyword for terminating voice input, the recognized character is displayed as shown in step S21. Do not display balloon images corresponding to columns. Subsequently, the smartphone 1 outputs a voice guidance “starts the mail application” from the speaker 11 for the purpose of notifying the user that the end of the voice input has been accepted, and a balloon corresponding to the voice guidance. The image (S-6) is displayed on the execution screen 50. As shown in step S22, the smartphone 1 activates the e-mail application 9D and displays a mail creation screen 60 in which character strings corresponding to each of the destination, the subject, and the text are inserted on the display 2A.

  FIG. 4 and FIG. 5 are diagrams showing another example of processing when the voice input to the body field of the mail application (for example, the email application 9D) is terminated by detecting a predetermined keyword.

  As illustrated in FIG. 4, when the smartphone 1 detects an operation on the icon 40a arranged on the lock screen 40 via the touch screen 2B (step S31), the smartphone 1 displays an interactive interface execution screen 50 on the display 2A. It is displayed (step S32). At this time, as shown in step S <b> 12, the smartphone 1 outputs a voice guidance “Please tell me your business.” From the speaker 11, and displays a speech balloon image (S-11) corresponding to the voice guidance on the execution screen. 50.

  Subsequently, when a voice is input from the microphone 8, the smartphone 1 executes a process for recognizing the input voice. When the recognition is successful, the smartphone 1 determines whether there is an application corresponding to the recognition result. When there is an application corresponding to the recognition result, the smartphone 1 determines whether to perform voice control related to the application. When the smartphone 1 performs voice control related to the application, the smartphone 1 responds to the user and starts voice control related to the application (steps S33 to S34).

  Specifically, when the smartphone 1 recognizes that the character string input by voice through the microphone 8 is “Mail to Mr. Ichiro Patent”, a speech bubble corresponding to the recognized character string as shown in step S33. The image (U-11) is displayed on the execution screen 50. Subsequently, the smartphone 1 detects an e-mail application 9D corresponding to the recognition result from the applications stored in the storage 9, and determines that voice control related to the e-mail application 9D is executed. Subsequently, as shown in step S <b> 34, the smartphone 1 starts voice control of the email application 9 </ b> D and displays an image 50 a for notifying the user on the execution screen 50. As shown in step S <b> 34, the smartphone 1 outputs a voice guidance “Please tell me the subject.” From the speaker 11 and displays a speech balloon image (S-12) corresponding to the voice guidance on the execution screen 50. .

  Subsequently, when a voice is input from the microphone 8, the smartphone 1 executes a process for recognizing the input voice. When the recognition is successful, the smartphone 1 stores the recognition result and responds to the user (steps S35 to S36).

  Specifically, when the smartphone 1 recognizes that the character string input by voice through the microphone 8 is “Good morning”, a speech balloon corresponding to the recognized character string is displayed as shown in step S35. The image (U-12) is displayed on the execution screen 50. Subsequently, the smartphone 1 stores the recognized character string, and displays the image 50b in which the recognized character string is inserted in the field corresponding to the subject of the image 50a in which the subject has not been input (step S36). After obtaining the subject of the mail, as shown in step S36, the smartphone 1 outputs a voice guidance from the speaker 11 “Please tell me the text. The corresponding balloon image (S-13) is displayed on the execution screen 50.

  Subsequently, when a voice is input from the microphone 8, the smartphone 1 executes a process for recognizing the input voice. If the recognition is successful, the smartphone 1 saves the recognition result and responds to the user on the condition that the recognition result is not the end keyword of the voice input or that the number of times of voice input has not reached the threshold. (Step S37 to Step S38).

  Specifically, when the smartphone 1 recognizes that the character string input by voice through the microphone 8 is “8 o'clock set”, a speech bubble corresponding to the recognized character string as shown in step S37. The image (U-13) is displayed on the execution screen 50. Subsequently, the smartphone 1 stores the recognized character string, and displays the image 50c in which the recognized character string is inserted in the field corresponding to the text of the image 50b in which the text has not been input (step S38). After obtaining the character string to be inserted into the body of the mail, as shown in step S38, the smartphone 1 again receives a voice guidance from the speaker 11 “Tell me the body. While outputting, the balloon image (S-14) corresponding to the voice guidance is displayed on the execution screen 50.

  Subsequently, when a voice is input from the microphone 8, the smartphone 1 executes a process for recognizing the input voice. If the recognition is successful, the smartphone 1 saves the recognition result and responds to the user on the condition that the recognition result is not the end keyword of the voice input or that the number of times of voice input has not reached the threshold. (Step S39 to Step S40).

  Specifically, when the smartphone 1 recognizes that the character string input by voice through the microphone 8 is “management.”, As shown in step S39, an image of a balloon corresponding to the recognized character string ( U-14) is displayed on the execution screen 50. In step S39, for example, even if the user intends to input “complete” by voice, if the smartphone 1 recognizes “management”, the process is continued as it is. Subsequently, the smartphone 1 stores the recognized character string, and displays the image 50d in which the recognized character string is inserted in the field corresponding to the text of the image 50c (step S40). The character string recognized by the smartphone 1 in step S39 and newly displayed in step S40 cannot be deleted by voice input. After re-acquiring the character string to be inserted into the body of the mail, as shown in step S40, the smartphone 1 once again receives the voice guidance “Tell me the body. , And a speech balloon image (S-15) corresponding to the voice guidance is displayed on the execution screen 50.

  Subsequently, when a voice is input from the microphone 8, the smartphone 1 executes a process for recognizing the input voice. If the recognition is successful, the smartphone 1 terminates the speech input and starts the application on the condition that the recognition result is an end keyword of speech input or the number of times of speech input is equal to or greater than a threshold value (step S41). -Step S42).

  Specifically, when the smartphone 1 recognizes that a character string input by voice through the microphone 8 is “complete”, which is a keyword for ending voice input, an image of a speech bubble corresponding to the recognized character string Is not displayed. Subsequently, as shown in step S <b> 41, the smartphone 1 outputs a voice guidance “starting the mail application” from the speaker 11 for the purpose of notifying the user that the end of the voice input has been accepted. The balloon image (S-16) corresponding to the voice guidance is displayed on the execution screen 50. As shown in step S42, the smartphone 1 activates the e-mail application 9D and displays a mail creation screen 60 in which character strings corresponding to the destination, subject, and body are inserted on the display 2A.

  As shown in step S39, even if the character string input by the user is not correctly recognized by the smartphone 1 as an end keyword, the character string input by the user is ended as shown in step S42. At the timing when the smartphone 1 correctly recognizes the keyword, the voice input to the text of the email application 9D is surely terminated.

  6 and 7 are diagrams illustrating an example of processing in the case where the voice input to the body field of the mail application (for example, the email application 9D) is terminated when the number of voice inputs is equal to or greater than the threshold value. .

  As shown in FIG. 6, when the smartphone 1 detects an operation on the icon 40a arranged on the lock screen 40 via the touch screen 2B (step S51), the smartphone 1 displays an interactive interface execution screen 50 on the display 2A. Displayed (step S52). At this time, as shown in step S52, the smartphone 1 outputs the voice guidance “Please tell me your business.” From the speaker 11 and displays the speech balloon image (S-21) corresponding to the voice guidance on the execution screen. 50.

  Subsequently, when a voice is input from the microphone 8, the smartphone 1 executes a process for recognizing the input voice. When the recognition is successful, the smartphone 1 determines whether there is an application corresponding to the recognition result. When there is an application corresponding to the recognition result, the smartphone 1 determines whether to perform voice control related to the application. When the smartphone 1 executes voice control related to the application, the smartphone 1 responds to the user and starts voice control related to the application (steps S53 to S54).

  Specifically, when the smartphone 1 recognizes that the character string input by voice through the microphone 8 is “Mail to Mr. Ichiro Patent”, a speech bubble corresponding to the recognized character string as shown in step S53. The image (U-21) is displayed on the execution screen 50. Subsequently, the smartphone 1 detects an e-mail application 9D corresponding to the recognition result from the applications stored in the storage 9, and determines that voice control related to the e-mail application 9D is executed. Subsequently, as shown in step S54, the smartphone 1 starts voice control of the email application 9D, and displays an image 50a for notifying the user on the execution screen 50. As shown in step S54, the smartphone 1 outputs the voice guidance “Please tell me the subject” from the speaker 11 and displays a speech balloon image (S-22) corresponding to the voice guidance on the execution screen 50. .

  Subsequently, when a voice is input from the microphone 8, the smartphone 1 executes a process for recognizing the input voice. When the recognition is successful, the smartphone 1 stores the recognition result and responds to the user (steps S55 to S56).

  Specifically, when the smartphone 1 recognizes that the character string input by voice through the microphone 8 is “Good morning”, a speech balloon corresponding to the recognized character string is displayed as shown in step S55. The image (U-22) is displayed on the execution screen 50. Subsequently, the smartphone 1 stores the recognized character string, and displays the image 50b in which the recognized character string is inserted in the field corresponding to the subject of the image 50a in which the subject has not been input (step S56). After obtaining the subject of the mail, as shown in step S56, the smartphone 1 outputs a voice guidance from the speaker 11 "Please tell me the text. The corresponding balloon image (S-23) is displayed on the execution screen 50.

  Subsequently, when a voice is input from the microphone 8, the smartphone 1 executes a process for recognizing the input voice. If the recognition is successful, the smartphone 1 saves the recognition result and responds to the user on the condition that the recognition result is not the end keyword of the voice input or that the number of times of voice input has not reached the threshold. (Step S57 to Step S58).

  Specifically, when the smartphone 1 recognizes that the character string input by voice through the microphone 8 is “8 o'clock set”, a speech bubble corresponding to the recognized character string as shown in step S57. The image (U-23) is displayed on the execution screen 50. Subsequently, the smartphone 1 stores the recognized character string, and displays the image 50c in which the recognized character string is inserted in the field corresponding to the text of the image 50b in which the text has not been input (step S58). After acquiring the character string to be inserted into the body of the mail, as shown in step S58, the smartphone 1 again receives a voice guidance from the speaker 11 “Tell me the body. While outputting, the balloon image (S-24) corresponding to the voice guidance is displayed on the execution screen 50.

  Subsequently, when a voice is input from the microphone 8, the smartphone 1 executes a process for recognizing the input voice. If the recognition is successful, the smartphone 1 saves the recognition result and responds to the user on the condition that the recognition result is not the end keyword of the voice input or that the number of times of voice input has not reached the threshold. (Steps S59 to S60).

  Specifically, when the smartphone 1 recognizes that the character string input by voice through the microphone 8 is “management.”, As shown in step S59, the image of the speech balloon corresponding to the recognized character string ( U-24) is displayed on the execution screen 50. In step S59, for example, even if the user intends to input “complete” by voice, when the smartphone 1 recognizes “management”, the process is continued as it is. Subsequently, the smartphone 1 stores the recognized character string, and displays the image 50d in which the recognized character string is inserted in the field corresponding to the text of the image 50c (step S60). After re-acquisition of the character string to be inserted into the body of the mail, as shown in step S60, the smartphone 1 again receives a voice guidance “Please tell me the body. , And a speech balloon image (S-25) corresponding to the voice guidance is displayed on the execution screen 50.

  Subsequently, when a voice is input from the microphone 8, the smartphone 1 executes a process for recognizing the input voice. If the recognition is successful, the smartphone 1 saves the recognition result and responds to the user on the condition that the recognition result is not the end keyword of the voice input or that the number of times of voice input has not reached the threshold. (Steps S61 to S62).

  Specifically, when the smartphone 1 recognizes that the character string input by voice through the microphone 8 is “management.”, As shown in step S61, an image of a balloon corresponding to the recognized character string ( U-25) is displayed on the execution screen 50. In step S61, for example, even if the user intends to input “complete” by voice, if the smartphone 1 recognizes “management”, the process is continued as it is. Subsequently, the smartphone 1 stores the recognized character string and displays the image 50e in which the recognized character string is inserted in the field corresponding to the text of the image 50d (step S62). After re-acquisition of the character string to be inserted into the body of the mail, as shown in step S62, the smartphone 1 once again receives the voice guidance “Tell me the body. And an image of a balloon (S-26) corresponding to the voice guidance is displayed on the execution screen 50.

  Subsequently, when a voice is input from the microphone 8, the smartphone 1 executes a process for recognizing the input voice. If the recognition is successful, the smartphone 1 saves the recognition result and responds to the user on the condition that the recognition result is not the end keyword of the voice input or that the number of times of voice input has not reached the threshold. (Step S63 to Step S64).

  Specifically, when the smartphone 1 recognizes that the character string input by voice via the microphone 8 is “management.”, As shown in step S63, an image of a balloon corresponding to the recognized character string ( U-26) is displayed on the execution screen 50. In step S63, for example, even if the user intends to input “complete” by voice, when the smartphone 1 recognizes “management”, the processing is continued as it is. Subsequently, the smartphone 1 stores the recognized character string and displays the image 50f in which the recognized character string is inserted in the field corresponding to the body of the image 50e (step S64). After re-acquiring the character string to be inserted into the body of the mail, as shown in step S64, the smartphone 1 again gives the voice guidance “Please tell me the body. And an image of a balloon (S-27) corresponding to the voice guidance is displayed on the execution screen 50.

  Subsequently, when a voice is input from the microphone 8, the smartphone 1 executes a process for recognizing the input voice. If the recognition is successful, the smartphone 1 ends the voice input and starts the application on the condition that the recognition result is a voice input end keyword or the number of times of voice input is equal to or greater than a threshold (step S65). -Step S66).

  Specifically, the smartphone 1 recognizes that the character string input by voice through the microphone 8 is “management”, and the number of input (accumulation) of the voice by inputting the voice corresponding to the corresponding character string. Is determined to be greater than or equal to the threshold value, the balloon image corresponding to the recognized character string is not displayed. Subsequently, as shown in step S <b> 65, the smartphone 1 outputs a voice guidance “Starting the mail application” from the speaker 11 for the purpose of notifying the user that the end of the voice input has been accepted. The balloon image (S-28) corresponding to the voice guidance is displayed on the execution screen 50. As shown in step S66, the smartphone 1 activates the e-mail application 9D, and displays a mail creation screen 60 in which character strings corresponding to the destination, the subject, and the text are inserted on the display 2A. As shown in step S66, each character string (management) corresponding to the recognition result is inserted into the mail creation screen 60 even if the recognition result of the smartphone 1 in steps S59 to S63 is incorrect.

  As shown in step S59 to step S63, even if the character string input by the user is not correctly recognized as the end keyword by the smartphone 1 many times, as shown in step S66, the number of times the user has input the voice. The voice input to the text of the e-mail application 9D surely ends at the timing when becomes equal to or greater than the threshold.

  An example of a processing procedure performed by the smartphone 1 according to the embodiment will be described with reference to FIGS. 8 and 9. FIG.8 and FIG.9 is a flowchart which shows an example of the process sequence by the smart phone 1 which concerns on embodiment. The processing procedures shown in FIGS. 8 and 9 are realized by the controller 10 executing the control program 9A stored in the storage 9.

  FIG. 8 shows the overall flow of voice control executed by the smartphone 1. As shown in FIG. 8, the controller 10 causes the display screen 2A to display an interactive interface execution screen (step S101). For example, when the controller 10 detects an operation on the icon 40a for activating the interactive interface via the touch screen 2B, the controller 10 activates the interactive interface and displays the interface execution screen 50 on the display 2A (see FIG. (See FIGS. 2 and 4).

  Subsequently, when acquiring the voice data of the voice input from the microphone 8 (step S102), the controller 10 executes a voice data recognition process to determine whether the voice has been successfully recognized (step S103).

  As a result of the determination, if the speech has been successfully recognized (Yes in step S103), the controller 10 determines whether an application corresponding to the recognition result has been detected from the applications stored in the storage 9 ( Step S104).

  If the controller 10 can detect an application corresponding to the recognition result as a result of the determination (step S104, Yes), the controller 10 determines whether to execute voice control related to the application corresponding to the recognition result (step S105).

  As a result of the determination, if the controller 10 performs voice control related to the application corresponding to the recognition result (Yes in step S105), the controller 10 starts voice control related to the application corresponding to the recognition result (step S106).

  When the voice control related to the application is finished, the controller 10 activates the application detected in step S104, reflects the result of the voice control in step S106 for the application (step S107), and the processing procedure shown in FIG. Exit. For example, the controller 10 activates the e-mail application 9D, and inserts the data of the subject and the text string of the text input through the interactive interface into the corresponding fields in the e-mail application 9D.

  In step S105, if the controller 10 does not execute voice control related to the application corresponding to the recognition result (No in step S105), the controller 10 executes the application detected in step S104 (step S108), and performs the processing shown in FIG. End the procedure.

  In step S104, when the controller 10 cannot detect an application corresponding to the recognition result as a result of the determination (step S104, No), the processing procedure shown in FIG. At this time, the controller 10 may display a list of applications stored in the storage 9 on the display 2A.

  In step S103, when the recognition fails as a result of the determination (step S103, No), the controller 10 determines whether or not the number of recognition failures exceeds a threshold (step S109). As a result of the determination, if the number of recognition failures exceeds the threshold (S109, Yes), the controller 10 ends the processing procedure shown in FIG. On the other hand, if the number of times of recognition failure does not exceed the threshold value as a result of the determination (S109, No), the controller 10 requests voice re-input by voice guidance and images, for example. (Step S110), the procedure proceeds to Step S102.

  FIG. 9 shows a flow of voice control related to the application. FIG. 9 corresponds to the procedure of step S106 shown in FIG. As shown in FIG. 9, the controller 10 requests voice input for the application that has decided to execute voice control in step S <b> 105 of FIG. 8 (step S <b> 201), and acquires voice data (step S <b> 202). . For example, the controller 10 requests voice input of a character string to be input to a field corresponding to the text of the email application 9D, and acquires a voice corresponding to the character string input to the text.

  Subsequently, the controller 10 executes speech data recognition processing in response to the request in S201, and determines whether or not the speech has been successfully recognized (step S203).

  As a result of the determination, if the speech is successfully recognized (step S203, Yes), the controller 10 determines whether or not the speech corresponding to the recognition result is a speech input end keyword (step S204). For example, the controller 10 determines whether or not the voice corresponding to the recognition result is “complete”.

  If the result of the determination is that the keyword is not the voice input end keyword (step S204, No), then, the number of voices (cumulative) input after starting the processing procedure shown in FIG. Is determined (step S205).

  If the result of determination is that the number of input voices is not greater than or equal to the threshold value (step S205, No), the controller 10 obtains the character string data to be inserted into the input field of the application detected in the process of FIG. Accumulate the acquired audio data. For example, the controller 10 accumulates the voice data acquired in step S202 as character string data to be input to the field corresponding to the text of the email application 9D.

  Subsequently, the controller 10 returns to the procedure of step S201 and continues the control shown in FIG.

  In step S205, if the result of determination is that the (accumulated) number of input voices since the start of the processing procedure shown in FIG. 9 is greater than or equal to the threshold (step S205, Yes), the controller 10 shows in FIG. The processing procedure ends.

  In step S204, when the voice corresponding to the recognition result is the voice input end keyword (step S204, Yes), the controller 10 ends the processing procedure shown in FIG.

  In step S203, if the result of determination is that the voice has not been successfully recognized (No in step S203), the controller 10 invalidates the voice input, returns to the procedure in step S201, and re-inputs the voice. Request.

  The order of the processing procedures shown in FIG. 9 is not limited to the example shown in FIG. In the processing procedure illustrated in FIG. 9, for example, the order of the procedure of step S204 and the procedure of step S205 may be reversed.

  As described above, the smartphone 1 according to the embodiment ends the voice input of the character string to be inserted into the input field of the application selected by voice through the interactive interface by voice control. For example, the smartphone 1 can end voice input of a character string to be inserted into a field corresponding to the text of the email application 9D by detecting an end keyword “complete”. Or the smart phone 1 can be complete | finished when the audio | voice input frequency becomes more than a threshold value (for example, 5 times), and the audio | voice input of the character string inserted in the field corresponding to the text of the email application 9D. As described above, the smartphone 1 according to the present embodiment enables a plurality of voice inputs to an input field in which a relatively long character string is input, for example, the body of an email. The voice input can be terminated by detecting the number of times or the number of times the voice is input. That is, the smartphone 1 according to the present embodiment can realize the start and end of input of a long sentence for a predetermined input field of the application only by voice control via the interactive interface, and improves processing via the interactive interface. be able to.

  In the above embodiment, when the smartphone 1 determines that the number of times of voice input is five, the voice input of the character string to be inserted into the field corresponding to the text of the email application 9D is terminated (FIGS. 6 and 7). However, any number of times can be applied to terminate the voice input.

  In the above-described embodiment, an example has been described in which the smartphone 1 terminates voice input of a character string to be inserted into a field corresponding to the text of the email application 9D by voice control via an interactive interface. However, the present invention is not limited to this, and the above-described implementation is generally performed for applications that perform voice input of character strings, such as voice input of character strings to the memo pad application 9E and voice input of character strings to the application that manages the schedule. The process (FIGS. 2-9, etc.) of the smart phone 1 which concerns on a form is applicable.

  In the example shown in FIGS. 5 and 7, even when the recognition result of the smartphone 1 for voice input is not intended by the user (in the case of an error), the character string corresponding to the recognition result (for example, management) The case where is inserted into the mail creation screen 60 has been described. However, the present invention is not limited to this, and the smartphone 1 determines that it is a character string related to misrecognition when a plurality of synonymous character strings are consecutively included in a voice-input character string, When starting up, it may not be inserted into the input field of the application. Alternatively, the smartphone 1 includes a list of character strings similar to the end keyword (for example, completion) in advance, and if there is a character string similar to the end keyword in the character string input by voice, the application When starting up, the character string may not be inserted into the input field of the application.

  In the embodiment described above, the smartphone 1 may end the voice input when it is determined that the recognition result for the voice input is the same as the end keyword (for example, bureaucracy).

  The characterizing embodiments have been described in order to fully and clearly disclose the technology according to the appended claims. However, the appended claims should not be limited to the above-described embodiments, but all modifications and alternatives that can be created by those skilled in the art within the scope of the basic matters shown in this specification. Should be embodied by a possible configuration.

  For example, each program shown in FIG. 1 may be divided into a plurality of modules. Alternatively, each program shown in FIG. 1 may be combined with another program.

  In the above embodiment, a smartphone has been described as an example of a device including a touch screen, but the device according to the appended claims is not limited to a smartphone. The device according to the appended claims may be an electronic device other than a smartphone. Electronic devices are, for example, mobile phones, tablets, portable personal computers, digital cameras, media players, electronic book readers, navigators, and game machines. Alternatively, the device according to the appended claims may be a stationary electronic device. The stationary electronic devices are, for example, a desktop personal computer and a television receiver.

DESCRIPTION OF SYMBOLS 1 Smart phone 2 Touch screen display 2A Display 2B Touch screen 3 Button 4 Illuminance sensor 5 Proximity sensor 6 Communication unit 7 Receiver 8 Microphone 9 Storage 9A Control program 9B Call application 9C SMS application 9D E-mail application 9E Notepad application 9F Movie application 9G Browser Application 9Y Phonebook data 9Z Setting data 10 Controller 11 Speaker 12 Camera 15 Attitude detection unit 18 Vibrator

Claims (4)

With a microphone,
A controller that performs voice control based on voice input through the microphone;
The controller is an electronic device that terminates voice input of a character string to be inserted into an input field of an application selected by the voice control by the voice control.   The electronic device according to claim 1, wherein the controller inserts the character string into the input field when the voice input is completed. A control method for controlling an electronic device having a microphone,
Performing voice control based on voice input via the microphone;
A step of terminating voice input of a character string to be inserted into an input field of an application selected by the voice control by the voice control. To an electronic device with a microphone
Performing voice control based on voice input via the microphone;
And a step of ending the voice input of the character string to be inserted into the input field of the application selected by the voice control by the voice control.

Images