JP2015076038A - Information processing method, information processing apparatus, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an information processing method, information processing apparatus, and program for improving the convenience of the execution of an instruction through voice recognition.SOLUTION: There is provided an information processing method for executing a predetermined command, which is specified on the basis of predetermined text information recognized through input voice information, the information processing method including a first mode of receiving the input of voice information and displaying the capability of receiving the input of voice information in a first area A1 on a screen, and a second mode of executing the predetermined command and displaying a result of the execution in a second area A2 larger than the first area on the screen.

Description

  The present invention relates to an information processing method, an information processing apparatus, and a program.

  In recent years, an interactive operation support system that supports input of user commands to electronic devices such as a television receiver and a personal computer has been developed (see, for example, Patent Document 1).

  The invention described in Patent Document 1 is an invention related to “interactive operation support system, interactive operation support method, and storage medium”, and specifically, “an anthropomorphic assistant that performs speech synthesis and animation reaction” By using the animation of the character called as a user interface, it is possible to provide a familiarity to the user and at the same time provide a response to a complicated command and an entrance to a service. Since the command system is provided, the user can easily operate the device with the same feeling as in a normal conversation.

JP 2002-41276 A

However, in the technique described in Patent Document 1 described above, no special consideration is given to the relationship between the display in the instruction reception state by voice and the display in the instruction execution state. Specifically, according to the technique described in Patent Document 1, the assistant is displayed on the full screen in the command reception state, and when the received command is executed, the result is displayed on the full screen. For this reason, the user cannot visually recognize the information processing contents other than the assistant in the command acceptance state. In the instruction execution state, it is difficult to grasp the relationship between the voice instruction and the execution result screen.
Accordingly, an object of the present invention is to provide an information processing method, an information processing apparatus, and a program that improve the usability of instruction execution by voice recognition.

  In order to solve the above-mentioned problem, the invention according to claim 1 is an information processing method for executing a predetermined command specified based on predetermined text information recognized from input voice information, wherein the voice A first mode for accepting the input of information and displaying in the first area on the screen that the input of the voice information can be accepted; and executing the predetermined command and displaying the execution result on the screen. And a second mode for displaying in a second area larger than the first area.

  ADVANTAGE OF THE INVENTION According to this invention, provision of the information processing method, information processing apparatus, and program which improve the usability of the command execution by speech recognition is realizable.

It is a block diagram of a personal computer as an information processing apparatus according to an embodiment. FIG. 2 is an example of a block diagram of a main part of the personal computer shown in FIG. 1. 3 is a flowchart showing an example of the operation of the personal computer shown in FIG. It is a figure which shows the state which the user has instruct | indicated to the personal computer with the sound. It is a conceptual diagram which shows the transition outline from starting of the personal computer shown in FIG. 1 to input standby, conversation, timeout, and session startup standby.

Next, an embodiment will be described.
<Configuration>
FIG. 1 is a block diagram of a personal computer as an information processing apparatus according to an embodiment.
1 includes a microphone 101, amplification circuits 102 and 104, a speaker 103, a display device 105, a keyboard 106, a mouse 107, an optical reader 108, a control means 109, an HDD (Hard Disk Drive). ) 110, a network connection unit 111, an I / O (Input / Output) 112, and a bus line 113.

The microphone 101 has a function of converting a user's voice into an electrical signal. Examples of the microphone 101 include a condenser microphone, but a dynamic microphone may be used.
The amplifier circuit 102 is a circuit that amplifies the electric signal from the microphone 101.
The speaker 103 has a function of converting an electrical signal into sound. The speaker 103 mainly has a function of transmitting the utterance content of an avatar obtained by anthropomorphizing a PC to a user.
The amplifier circuit 104 is a circuit that amplifies the audio signal to a level for driving the speaker 103.
The display device 105 has a function of displaying an image, characters, and the like including a balloon that displays the avatar and the utterance contents of the avatar as characters. Examples of the display device 105 include a liquid crystal display element. The display device 105 displays in the first area A1 on the screen (see FIG. 5 described later) that it can accept input of audio information. On the display device 105, the execution result of the predetermined command is displayed in the second area A2 (see FIG. 5) on the screen. The first area A1 may be a bar-shaped area sharing a side with at least one side of a rectangular screen. The second area A2 may be the entire area of the screen.
The keyboard 106 is an input device for inputting characters, numbers, and symbols.
The mouse 107 is a kind of input device, and has a function of moving the cursor of the display device 105 by moving on the desk.
The optical reader 108 has a function of reading an optical medium such as a CD (Compact Disk), a DVD (Digital Versatile Disc), or a CD-R (Compact Disc-Recordable).

The control means 109 is an element having an overall control function and a voice processing function for the PC 100, and includes, for example, a CPU (Central Processing Unit). The voice processing function is a function for outputting, analyzing, and synthesizing mainly input voice as text data.
The control unit 109 includes an input control unit 109a, a voice recognition unit 109b, a voice analysis unit 109c, a search unit 109d, a voice synthesis unit 109e, and a display device control unit 109f, each of which is configured by software.

The input control unit 109a performs processing based on a command obtained by analyzing a signal obtained by converting the sound input to the microphone 101, key input from the keyboard 106, and click or drag from the mouse 107. Has a function of converting a signal such as a character display, a numerical display, a sign display, a cursor movement, a command, and the like.
The voice recognition unit 109b is a client type voice recognition unit 203 described later.
The voice analysis unit 109c is a voice signal interpretation unit 202 described later.
The search means 109d is means for searching the Internet via the network 207. The search unit 109d has a function of searching for a keyword by connecting to a network with a preset browser and connecting to a preset Internet search service company when a search instruction is received from a user.
The voice synthesizer 109e is a client type voice synthesizer 210 described later.

The display device control unit 109f has a function of selecting, as the region A1 or the region A2, a region where an avatar, a speech balloon, a command execution result, and the like to be displayed on the display device 105 are displayed on the screen regarding the speech recognition process. That is, when the PC 100 is in a voice input standby state (first mode), the display device control unit 109f displays that voice input can be received in the first area A1. The indication that the voice input can be accepted can be displayed by displaying an avatar and a speech balloon stating that “speech input can be accepted” in a stick-shaped area, but is not limited thereto. In addition, during a conversation, that is, when the input of voice information and the execution of a command specified by the voice information are continuous within a predetermined time (second mode), the command is executed in the second area A2. Display the results. In the second area A2, in addition to the execution result of the command, an avatar, an avatar balloon, a bar-shaped area, a guide icon or a description of the guide icon may be displayed together. If supplementary information on the command execution result is described in the balloon of the avatar, the user can grasp the command execution result more accurately.
For example, the avatar balloon may characterize the response to the question from the user by the setting of the PC 100 or may be left blank.
Other functions of the display device control unit 109f correspond to a client application unit 204 and a local content unit 208 described later.

  The HDD 110 is a kind of storage device, and has a ROM (Read Only Memory) area and a RAM (Random Access Memory) area. The ROM area is an area for storing a control program, and the RAM area is an area used as a memory.

The network connection unit 111 is a known device having a function of connecting to an external server via the network 207. Either wireless or wired means may be used.
The I / O 112 is an input / output device having a function of connecting an external electronic device such as a USB (Universal Serial Bus line) flash memory or a printer.

FIG. 2 is an example of a block diagram of the main part of the PC shown in FIG.
In FIG. 2, the PC 100 according to the embodiment of the present invention converts the voice of the user 200 input from the microphone 101 into voice data (electrical signal), and the voice data is interpreted by the voice signal interpretation unit 202. Is recognized by the client-type speech recognition unit 203. The client type voice recognition unit 203 passes the recognized voice data to the client application unit 204.

  The client application unit 204 checks whether an answer to the inquiry from the user 200 is stored in the local content unit 208 in an offline state. If the answer is stored in the local content unit 208, the client application unit 204 An answer to the inquiry is output from the speaker 103 via the text reading unit 209 and the client-type speech synthesizer 210 described later.

  If the answer to the inquiry from the user 200 is not stored in the local content unit 208, it means that the PC 100 alone does not have an answer, so the network connection unit 206 connected to the network 207 such as the Internet is used. Then, an answer to the inquiry from the user is searched using a search engine on the Internet, and the obtained search result is output as voice from the speaker 103 via the text reading unit 209 and the client-type speech synthesizer 210.

  The client application unit 204 converts the answer obtained from the local content unit 208 or the network 207 into text (character) data and passes it to the text reading unit 209. The text reading unit 209 reads the text data and passes it to the client-type speech synthesizer 210. The client-type voice synthesizer 210 synthesizes voice data with voice data that can be recognized by a human and passes the voice data to the speaker 103. The speaker 103 converts sound data (electrical signal) into sound. In addition, in accordance with the sound emitted from the speaker 103, detailed information related to the sound is displayed on the display device 105.

  Next, screen display at the time of starting the information processing apparatus according to the embodiment of the present invention will be described. 3 to 5 are diagrams for explaining screen display when the information processing apparatus according to the embodiment of the present invention is started.

  The avatar of PC100 which concerns on embodiment of this invention can perform various greetings according to the time zone and day of the week at the time of starting. For example, when the activation time is in the morning time zone, the avatar shown in FIG. 5 displays related information on the display device 105 (FIG. 1) as it says “Good morning!”. Similarly, when you start the avatar during the day, to say "Hello!". In addition to the time zone, utterances according to the days of the week such as weekdays and holidays can be performed.

<Operation>
FIG. 3 is a flowchart showing an example of the operation of the PC shown in FIG. FIG. 4 is a diagram illustrating a state in which the user instructs the PC by voice. FIG. 5A shows a state before the instruction from the user when the voice recognition processing is in operation among the screens displayed on the display device of the PC shown in FIG. 4, and FIG. It is a figure which shows the state after the instruction | indication from a user while the speech recognition process is operating.

In FIG. 3, the subject of operation is a control means.
When receiving an instruction to start a series of processes for executing a command by voice recognition, first, the control unit 109 transitions to a first mode in which a voice recognition start instruction can be accepted, and can accept a voice recognition start instruction. This is displayed in the area A1 of the display device 105 (step S1). In the area A1, in addition to being able to accept voice input, an avatar may be displayed, and the fact that voice input can be accepted may be displayed in a speech bubble as speech content of the avatar. The area A1 can be displayed as a task bar that shares at least one side of the rectangular screen in the display device 105, but is not limited thereto.
Subsequently, the control means 109 determines whether or not a voice recognition start instruction is input from the user (step S2). The voice recognition start instruction may be recognition by a simple voice recognition process of inputting a wakeup keyword (for example, “Sherri”) by a user's voice, or an operation to a predetermined icon or hardware switch displayed on the screen It may be.
The control means 109 stands by until a voice recognition start instruction is input from the user (step S2 / N). When a voice recognition start instruction is input from the user (step S2 / Y), the control means 109 Recognition processing is started (step S3).
Subsequently, the control unit 109 waits for a voice recognition stop instruction from the user (step S4). When a voice recognition stop instruction is input from the user (step S4 / Y), the process returns to step S1 after the voice recognition process is completed. The voice recognition stop instruction from the user may be determined by the absence of voice input for a predetermined time after the execution of the command instructed by voice, and the sleep word (for example, “bye-bye”) by the user's voice may be determined. The input may be recognized by voice recognition processing, or may be an operation on a predetermined icon or hardware switch displayed on the screen. Further, when the voice recognition process is ended, a message indicating that the voice recognition is to be ended may be displayed as a speech of the avatar in a balloon.
If no voice recognition stop instruction is input from the user (step S4 / N), the control means 109 determines whether or not there is a voice input instructing a command from the user (step S5).

When there is a voice input instructing a command from the user (step S5 / Y), the command specified by the voice is specified and executed, and the execution result of the command is displayed in an area A2 larger than the area A1 of the display device 105. (Step S6), the process returns to Step S4 and waits for a voice recognition stop instruction. The area A2 may be the entire display screen in the display device 105, but is not limited thereto. Further, an avatar may be displayed in the area A2 together with the execution result of the command, and supplementary information on the execution result of the command may be displayed as a speech of the avatar in a balloon. At this time, if the avatar is displayed in the area A2 at the same display scale as the avatar displayed in the area A1 in step S1, the user's uncomfortable feeling due to the transition between the first mode and the second mode is reduced. This is preferable.
Further, when there is a voice input instructing a command from the user, it is further determined whether or not the execution result of the instructed command requires screen display, and the execution result that requires screen display is determined. Sometimes, the step of displaying the result in the area A2 may be executed following step S5. In this way, for example, unnecessary changes in the display area of the screen can be prevented when executing commands such as volume adjustment and screen brightness adjustment, and user convenience is improved. In this step, when the command execution result does not require screen display, the process returns to step S4 in the same manner as after step S5.
If there is no voice input instructing a command from the user (step S5 / N), the process returns to step S4 to continue the input of the voice recognition stop instruction and the voice input instructing the command.

That is, as shown in FIG. 4, for example, it is assumed that the user 200 is making a seat while sitting on the dresser's chest, and the PC 100 that is performing the voice recognition operation is placed on the sofa. When a wake-up keyword as a keyword is issued from the user 200, a control unit as a determination unit determines and starts a response operation for an inquiry as a command.
When the user 200 makes “Sherry” to the PC 100 while making up, the voice of the speaker 103 of the PC 100 instantly says “Good morning. What do you want to do? Is emitted. At the same time, the area A2 is displayed on the monitor 100a. Although the PC 100 and the user 200 are separated from each other in FIG. 4, the user 200 can view the monitor 100 a of the PC 100 at any time. For this reason, since the size of the avatar is hardly changed, continuity of a series of modes can be obtained, and operation can be performed without a sense of incongruity.

FIG. 5 is a conceptual diagram showing an outline of the transition from activation of the personal computer 100 shown in FIG. 1 to input standby, conversation, timeout, and session activation standby.
<Startup>
The voice recognition function of the personal computer 100 shown in FIG. 1 is activated from a soft navigation (software navigator) or an application (application software). The soft navigation or application is activated by the user 200 using the mouse 107, the keyboard 106, or a touch panel (not shown).

<Active Waiting>
When the voice recognition function is activated, the first area A1 is displayed on the screen of the display device 105. The first area A1 includes an avatar 150, a balloon 151 of the avatar 150, and a bar-shaped area 152. In the bar-shaped area 152, an area 153 for keyboard input or mouse input is arranged.

<During conversation>
When the user 200 starts a voice conversation with the personal computer 100 in the input standby state, an area A2 larger than the area A1 is displayed. The size of the area A2 is the entire area of the screen. In the area A2, in addition to the avatar 150, the balloon 151 of the avatar 150, and the bar-shaped area 152, for example, an icon 155a meaning a restaurant, an icon 155b meaning a train, an icon 155c meaning weather, and an icon meaning transfer information 155d, various icons such as an icon 155e indicating a calendar, and descriptions 156a to 156e of the icons 155a to 155e are displayed in an itemized form.
The user 200 can make a question and a search request as if having a conversation with the avatar 150 of the personal computer 100. If the audio from the user 200 is interrupted even after a predetermined time has elapsed after the conversation is started, the screen display returns from the area A2 to the original area A1 (session timeout).

The area A1 indicating the input standby state or the area A2 indicating the state of talking is clicked or tapped on the keyboard 106, the mouse 107, or a touch panel (not shown) in the bar-shaped area 152 (taskbar or Iconized in the task tray. By double-tapping the icon, the session transitions to the session activation waiting state (Inactive Waiting) state.
In the above, according to this embodiment, since the size of the avatar is almost the same in the areas A1 and A2 having different sizes, the continuity of a series of modes is obtained. For this reason, there is no sense of incongruity in operation.

<Program>
The information processing apparatus according to the present invention described above is realized by a program that causes a computer to execute processing. Examples of the computer include a personal computer, but the present invention is not limited to this. Therefore, as an example, a case where the function of the present invention is realized by a program will be described below.

For example,
Input means for inputting speech information; recognition means for recognizing predetermined text information from the speech information input to the input means; and executing a predetermined command specified based on the recognized predetermined text information A computer-readable program of an information processing device comprising means,
The computer is
In the display means,
A procedure for displaying in the first area on the screen that the input of the audio information can be received in the first mode of receiving the input of the audio information;
Displaying the execution result of the predetermined command in a second area larger than the first area on the screen in the second mode for executing the predetermined command;
A program for executing

Thus, the information processing apparatus according to the present invention can be realized anywhere as long as there is a computer environment capable of executing the program.
Such a program may be stored in a computer-readable storage medium.

<Storage medium>
Here, examples of the storage medium include computer-readable storage media such as CD-ROM, flexible disk (FD), and CD-R, semiconductor memories such as flash memory, RAM, ROM, and FeRAM, and HDD.

  A flexible disk means Flexible Disk: FD. RAM is an abbreviation for Random-Access Memory. ROM is an abbreviation for Read-Only Memory. FeRAM is an abbreviation for Ferroelectric RAM and means a ferroelectric memory.

  In the above, according to the present invention, input means for inputting speech information, recognition means for recognizing predetermined text information from the speech information input to the input means, and identification based on the recognized predetermined text information. An information processing apparatus that executes a predetermined command, and displays in the first area on the screen that the voice information input and the voice information input can be accepted, A display means for executing a predetermined command and displaying an execution result in a second area on the screen is provided, and the continuity of a series of modes is obtained because the second area is larger than the first area. Information processing method, information processing apparatus, and program can be provided.

  The above-described embodiment shows an example of a preferred embodiment of the present invention, and the present invention is not limited thereto, and various modifications can be made without departing from the scope of the invention. is there. For example, although a young woman is used as an avatar in the present embodiment, the present invention is not limited to this, and may be a male or an animated character.

100 Personal computer (PC, information processing device)
100a monitor 101 microphone 102, 104 amplifier circuit 103 speaker 105 display device 106 keyboard 107 mouse 108 optical reader 109 control means 109a input control means 109b speech recognition means 109c speech analysis means 109d search means 109e speech synthesis means 109f display device control means 110 HDD
111 Network connection 112 I / O
113 Bus Line 150 Avatar 151 Speech Bubble 200 User 202 Speech Signal Interpretation Unit 203 Client Type Speech Recognition Unit 204 Client Application Unit 207 Network 208 Local Content Unit 209 Text Reading Top 210 Client Type Speech Synthesis Unit

Claims (8)

An information processing method for executing a predetermined command specified based on predetermined text information recognized from input speech information,
A first mode for accepting the input of the voice information and displaying in the first area on the screen that the voice information can be accepted, and executing the predetermined command and displaying the execution result on the screen And a second mode for displaying in a second area larger than the first area.   2. The information processing method according to claim 1, wherein when the predetermined command whose execution result is not displayed on the screen is executed in the second mode, the display area is not changed.   An avatar that is an anthropomorphic of an information processing device that performs information processing is displayed as if speaking that the input of the voice information can be accepted in the first mode, and in the second mode, The information processing method according to claim 1 or 2, wherein the supplementary information of the execution result is displayed as if it is spoken.   The information processing method according to claim 3, wherein the display scale of the avatar is common in the first mode and the second mode.   5. The information processing method according to claim 1, wherein the first area is a bar-shaped area sharing a side with at least one side of the screen having a rectangular shape.   The information processing method according to claim 1, wherein the second area is an entire area of the screen. Input means for inputting speech information; recognition means for recognizing predetermined text information from the speech information input to the input means; and executing a predetermined command specified based on the recognized predetermined text information An information processing apparatus comprising means,
In the first mode for accepting the input of the audio information, the fact that the input of the audio information can be accepted is displayed in the first area on the screen, and the predetermined command is executed in the second mode for executing the predetermined command. Display means for displaying a command execution result in a second area on the screen;
An information processing apparatus, wherein the second area is larger than the first area. Input means for inputting speech information; recognition means for recognizing predetermined text information from the speech information input to the input means; and executing a predetermined command specified based on the recognized predetermined text information A computer-readable program of an information processing device comprising means,
The computer is
In the display means,
A procedure for displaying in the first area on the screen that the input of the audio information can be received in the first mode of receiving the input of the audio information;
Displaying the execution result of the predetermined command in a second area larger than the first area on the screen in the second mode for executing the predetermined command;
A program for running

Images