JP2001216131A - Information processor, its method and program storage medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To allow a user to recognize which application receives a voice recognition result to be sent. SOLUTION: When an application for voice recognition is started and the display of a window for the application is instructed as minimum display, a small display window 241 for the application corresponds to voice recognition and is displayed in the vicinity of an activated electronic pet window 191. The window 241 is displayed only during the execution of voice recognition.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an information processing apparatus and method, and a program storage medium, and more particularly, to an information processing apparatus and method suitable for an apparatus for performing voice recognition.
And a program storage medium.

[0002]

2. Description of the Related Art Information processing apparatuses, such as personal computers, which recognize voice and execute predetermined processing in response to the recognized voice, are becoming widespread. For example, when voice recognition is executed in a personal computer, a voice recognition application is activated, and a window or the like indicating that the application is activated is displayed on a display.
The window displays the result of recognition of words spoken by the user as speech.

[0003]

A window displayed while the above-described speech recognition is being executed is sometimes displayed in a small size and stored in a task bar or the like. When such a small-sized display and a plurality of applications that support voice recognition are executed, the user has a problem that it is difficult to recognize to which application the recognition result of the input voice was sent. there were.

[0004] In the case of voice recognition, the user must
It is necessary to speak a predetermined command. The commands differ depending on the application and the number of commands is large, so that the user cannot remember them and cannot use them easily, so that there is a problem that usability is deteriorated.

The present invention has been made in view of such circumstances, and when a window is displayed in a small size,
It is an object of the present invention to realize a user-friendly voice recognition by displaying a window near an application to which a voice recognition result is sent and enabling a list of commands to be called up by voice.

[0006]

According to a first aspect of the present invention, there is provided an information processing apparatus comprising: first determining means for determining whether a voice recognition state is instructed; If it is determined that the recognition state is indicated,
A second determining means for determining whether or not a program for executing a predetermined process according to the result of the voice recognition is activated and in an active state; and a result of the voice recognition by the second determining means. When it is determined that the program for executing the predetermined process is activated and is in the active state in response to the above, the first window indicating the state in which the voice recognition is instructed is set to correspond to the program. First display control means for controlling display so as to be displayed near or overlapping the second window, and predetermined processing is executed by the second determination means in accordance with the result of voice recognition. If it is determined that the program has not been started or has been started but is not active, the first window is displayed at a predetermined position. Characterized in that it comprises a second display control means for controlling indicates.

If the result of the speech recognition indicates that a command list for executing a predetermined process is to be displayed, a third display control means for controlling the display so that the command list is displayed. May be further included.

According to a third aspect of the present invention, there is provided an information processing method comprising: a first determining step of determining whether a state of recognizing a voice is instructed; If it is determined that the instruction has been given, a second determination step of determining whether or not a program for executing a predetermined process according to the result of the voice recognition is activated and in an active state; If a program for executing a predetermined process corresponding to the result of speech recognition is started in the process of the second determination step and it is determined that the program is in an active state, a state in which speech recognition is instructed A first display control step of controlling the display so that a first window indicating the above is displayed near or overlapping with the second window corresponding to the program;
In the processing of the second determination step, the program for executing the predetermined processing corresponding to the result of the voice recognition has not been started,
Or a second display control step of controlling display so that the first window is displayed at a predetermined position when it is determined that the first window is activated but not in an active state. It is characterized by the following.

According to a fourth aspect of the present invention, a program in a program storage medium recognizes a voice by a first determining step of determining whether a voice recognition state is instructed, and a process of the first determining step. If it is determined that the state has been instructed, a second determination step of activating a program for executing a predetermined process in accordance with the result of voice recognition and determining whether the state is active When,
If a program for executing a predetermined process corresponding to the result of speech recognition is started in the process of the second determination step and it is determined that the program is in an active state, a state in which speech recognition is instructed Is displayed in the vicinity of the second window corresponding to the program, or
A first display control step of controlling display so as to be displayed at an overlapping position and a program for executing predetermined processing corresponding to a result of voice recognition in the processing of the second determination step have not been started, or A second display control step of controlling the display so that the first window is displayed at a predetermined position when it is determined that the first window is activated but not active. It is characterized by.

[0010] The information processing apparatus according to claim 1, claim 3,
In the information processing method described in the above, and the program storage medium described in the claim 4, it is determined that the program for executing the predetermined processing in response to the result of the voice recognition is activated and is in an active state. In this case, the first window indicating the state where the voice recognition is instructed is located near the second window displayed corresponding to the program,
Or, the display is controlled to be displayed at an overlapping position, and a program for executing a predetermined process corresponding to a result of voice recognition is not activated, or is activated but is not in an active state. If determined, the first
The display is controlled so that this window is displayed at a predetermined position.

[0011]

DESCRIPTION OF THE PREFERRED EMBODIMENTS One embodiment of an information processing apparatus according to the present invention will be described below with reference to the drawings.

1 to 6 show examples of the configuration of a portable personal computer to which the present invention is applied. The personal computer 1 is a mini-notebook type personal computer, and basically includes a main body 2 and a display unit 3 which can be opened and closed with respect to the main body 2. 1 is an external perspective view showing a state in which the display unit 3 is opened with respect to the main body 2, FIG. 2 is a plan view of FIG. 1, and FIG. 3 shows a state in which the display unit 3 is closed with respect to the main body 2. 4 is a right side view showing the display unit 3 opened 180 degrees with respect to the main body 2, FIG. 5 is a front view of FIG. 3, and FIG.
FIG.

The main body 2 is provided with a keyboard 4 operated when inputting various characters and symbols, and a stick-type pointing device 5 operated when moving a mouse cursor or the like. Also,
On the upper surface of the main body 2, there are further provided a speaker 8 for outputting sound and a shutter button 10 operated when capturing an image with the CCD video camera 23 provided on the display unit 3.

A claw 13 is provided at an upper end portion of the display unit 3. As shown in FIG. 3, when the display unit 3 is closed with respect to the main body 2, Is provided with a hole 6 into which the claw 13 is fitted. A slide lever 7 is provided on the front surface of the main body 2 so as to be movable in parallel with the front surface. The slide lever 7 engages with a claw 13 fitted in the hole 6 to lock and unlock. I can do it. By releasing the lock, the display unit 3 can be rotated with respect to the main body 2. Microphone 2 next to claw 13
4 is attached. As shown in FIG. 6, the microphone 24 can collect sound from the back.

A programmable power key (PPK) 9 is also provided on the front of the main body 2. As shown in FIG. 4, an exhaust hole 11 is provided on the right side surface of the main body 2, and an intake hole 14 is provided at a lower part of the front surface of the main body 2 as shown in FIG. 5. Further, a PCMCIA (Personal Computer Memory Card Internat) is provided on the right side of the exhaust hole 11.
A slot 12 for inserting an ional association) card (PC card) is provided.

An LCD for displaying an image is provided on the front of the display unit 3.
(Liquid Crystal Display) 21 is provided, and an imaging unit 22 is provided at the upper end thereof so as to be rotatable with respect to the display unit 3. That is, this imaging unit 22
It can rotate to any position within a range of 180 degrees between the same direction as D21 and the opposite direction (backward direction). A CCD video camera 23 is attached to the imaging unit 22.

A power lamp is provided on the lower body side of the display unit 3.
PL, battery lamp BL, message lamp ML, other LEDs
Is provided. Reference numeral 40 shown in FIG. 3 is a power switch provided on the left side surface of the main body 2, and reference numeral 25 shown in FIG. 5 is an adjustment ring for adjusting the focus of the CCD video camera 23. Further, FIG.
Reference numeral 26 denotes a lid for covering an opening for mounting an additional memory in the main body 2, and reference numeral 41 denotes a lid 26.
This is a small hole for inserting a pin for removing the lock claw.

FIG. 7 shows the internal configuration of the personal computer 1. As shown in FIG. 7, a CPU (Central Processing Unit) 52, a PC card 53 inserted as needed, and a RAM (Random Access M
emory) 54 and a graphic chip 81 are connected. The internal bus 51 is connected to an external bus 55. The external bus 55 includes a hard disk drive (HDD) 56, an I / O (input / output) controller 57, a keyboard controller 58, a stick pointing device controller 59, and a sound. Chip 60, LC
The D controller 83, the modem 50 and the like are connected.

The CPU 52 is a controller that controls each function, and the PC card 53 is appropriately mounted when an optional function is added.

In the RAM 54, an e-mail program (application program) 54A, an auto-pilot program (application program) 54B, and an OS (basic program) 54C are transferred from the HDD 56 and stored at the time of completion of the activation. You.

The electronic mail program 54A is a program for sending and receiving messages via a network from a communication line such as a telephone line. E-mail program 54
A has an incoming mail acquisition function as a specific function. This incoming mail acquisition function checks the mail server 93 to see if mail addressed to the user (user) has arrived in the mailbox 93A, and if there is mail addressed to the user, obtains the mail. Execute.

The auto-pilot program 54B includes a plurality of preset processes (or programs).
This is a program that is started and processed sequentially in a preset order.

The OS 54C controls the basic operation of a computer typified by Windows 98 (trademark).

On the other hand, a hard disk drive (HDD) 56 on the external bus 55 side has an electronic mail program 56
A, an autopilot program 56B, and an OS 56C are stored. OS in the hard disk drive 56
56C, the auto-pilot program 56B, and the e-mail program 56A are sequentially transferred and stored in the RAM 54 in the course of the startup (boot-up) process.

The I / O controller 57 has a microcontroller 61, and the microcontroller 61 is provided with an I / O interface 62. This microcontroller 61 includes an I / O interface 6
2, the CPU 63, the RAM 64, and the ROM 69 are connected to each other. The RAM 64 includes a key input status register 65 and an LED (light emitting diode) control register 6.
6, a set time register 67 and a register 68 are provided. The set time register 67 is used to start the operation of the start-up sequence control unit 76 at a time (start-up condition) set by the user in advance. The register 68 stores a correspondence between a preset operation key combination (start condition) and an application program to be started. When the stored operation key combination is input by the user, the register 68 stores the correspondence. The activated application program (for example, e-mail) is activated.

When the programmable power key (PPK) 9 for one-touch operation is pressed, the key input status register 65 stores an operation key flag. The LED control register 66 controls the lighting of a message lamp ML that indicates the activation state of the application program (e-mail) stored in the register 68. The set time register 67 can arbitrarily set a predetermined time.

A backup battery 74 is connected to the microcontroller 61.
The values of the registers 65, 66, and 67 are retained even when the power of the main unit 2 is turned off.

The ROM 69 in the microcontroller 61 stores a wake-up program 70, a key input monitoring program 71, and an LED control program 72 in advance. The ROM 69 is, for example, an EEPROM (electricall
y erasable and programmable read only memory). This EEPROM is also called a flash memory. Further, an RTC (Real-Time Clock) 75 that constantly counts the current time is connected to the microcontroller 61.

The wake-up program 70 in the ROM 69 checks, based on the current time data supplied from the RTC 75, whether or not the time set in the set time register 67 has been reached. , A program for starting a predetermined process (or a program). The key input monitoring program 71
This is a program for constantly monitoring whether or not K9 has been pressed by the user. The LED control program 72 is a program for controlling lighting of the message lamp ML.

The ROM 69 further includes a BIOS (Basic Input O
utput System) 73 is written. This BIOS 73
Has a function of activating the OS 56C when the power is turned on, and transmitting and receiving data between various application software and peripheral devices (display, keyboard, hard disk drive, etc.) after the activation.

A keyboard controller 58 connected to the external bus 55 controls an input from the keyboard 4. The stick pointing device controller 59 controls an input of the stick pointing device 5.

The sound chip 60 includes the microphone 24
Or an audio signal is supplied to the built-in speaker 8.

The modem 50 can be connected to a communication network 92 such as the Internet, a mail server 93, and the like via a public telephone line 90 and an Internet service provider 91.

After the image data captured by the CCD video camera 23 is processed by the processing unit 82, the graphics chip 81 connected to the internal bus 51
omed Video) port. The graphic chip 81 includes the processing unit 8
Video data input from the CCD video camera 23 via the VRAM 2 is stored in the built-in VRAM 81, read out as appropriate, and output to the LCD controller 83. The LCD controller 83 outputs the image data supplied from the graphic chip 81 to the LCD 21 for display. The backlight 84 illuminates the LCD 21 from behind.

The power switch 40 is operated when the power is turned on or off. The half-press switch 85 is turned on when the shutter button 10 is half-pressed, and the full-press switch 86 is turned on when the shutter button 10 is fully pressed. The reversing switch 87 is connected to the imaging unit 2
2 is rotated 180 degrees (CCD video camera 23
It is turned on when it is rotated in the direction of imaging the opposite side of the CD 21).

The drive 88 is connected to the external bus 55. The drive 88 includes a magnetic disk 351 (including a floppy disk) and an optical disk 352 (CD-ROM (Compa
ct Disc-Read Only Memory), DVD (Digital Versatile D)
isc)), magneto-optical disk 353 (MD (Mini-Dis
c)) or a semiconductor memory 354 or the like, and the mounted magnetic disk 351 and optical disk 35
2. Magneto-optical disk 353 or semiconductor memory 354
Program or data recorded in the HDD 56 or the R via the external bus 55 or the internal bus 51.
Supply to AM54.

The drive 88 is connected to the modem 50, the HDD 56, or the RAM 5 via the external bus 55 or the internal bus 51.
4 is recorded on the mounted magnetic disk 351, optical disk 352, magneto-optical disk 353, semiconductor memory 354, or the like.

FIG. 8 is a diagram showing functional blocks according to a predetermined program when the personal computer 1 starts a program relating to voice recognition. The speech recognition engine 101 is input from the microphone 24 based on the reading of the kanji stored in the reading kana dictionary database 111 in advance, or the recognition word or grammar stored in the engine recognition word / grammar database 112 in advance. Enter the data corresponding to the user's voice
The data in a predetermined format such as text corresponding to the words spoken by the user is generated and supplied to the voice commander 102.

The voice recognition engine 101 receives data such as a recognized word or grammar from the voice commander 102 and stores the data in the reading kana dictionary database 111 or the recognized word / grammar database 112 for the engine.

When data such as a word (text or the like) corresponding to a predetermined word spoken by the user is supplied from the voice recognition engine 101, the voice commander 102 executes the still image photographing program 103 and the still image browsing program 10
4 or start the electronic pet program 105,
Alternatively, a predetermined command (corresponding to a word spoken by the user) is transmitted to the still image photographing program 103, the still image browsing program 104, or the electronic pet program 105.

When the voice commander 102 receives data such as a word (text or the like) corresponding to another predetermined word spoken by the user from the voice recognition engine 101,
The e-mail program 54A, based on the setting related to the activation stored in the launcher setting database 113,
Activate the word processor program 106 or the spreadsheet program 107 and execute the e-mail program 54A.
Is supplied with predetermined data such as a mail address.

The voice commander 102 has a graphical user interface. Various settings are made by the user through the graphical user interface, and the contents set by the user are classified.
Application program (E-mail program 5
4A, the settings related to the activation of the word processor program 106 or the spreadsheet program 107) in the launcher setting database 113, the settings related to reading of kanji, or commands of the still image photographing program 103, the still image browsing program 104, or the electronic pet program 105 Is stored in the dictionary setting database 114, and the setting relating to the word or grammar to be recognized is stored in the recognition word / grammar database 115, respectively.

When the voice commander 102 causes the voice recognition engine 101 to recognize a voice at a predetermined timing, for example, the voice commander 102 converts the recognition word data and the grammar data stored in the recognition word / grammar database 115 into the voice recognition engine 101. Send to 101.

The speech recognition engine 101 reads the kana dictionary database 111 for the user based on the data inputted when the OS 54C is started to determine the user.
And the engine recognition word / grammar database 112 is used. The voice commander 102 determines the user based on data input when the OS 54C is started,
The launcher setting database 113, dictionary setting database 114, and recognized word / grammar database 115 for the user are used.

A kana dictionary database 111, an engine recognition word / grammar database 112, a launcher setting database 113, a dictionary setting database 114, and a recognition word / grammar database 115 are generated for each user of the personal computer 1 and recorded on the HDD 56. Is done.

The still image shooting program 103 converts the image input from the CCD video camera 23 into the shutter button 1
Based on a signal corresponding to an operation such as 0, data of a still image is generated and recorded in the HDD 56 as a predetermined file.

The still image browsing program 104 selects a file of the still image recorded by the still image photographing program 103 or allows the user to select the file, and causes the LCD 21 to display the selected still image. Electronic pet program 105
Displays a virtual pet on the LCD 21 and gives an instruction to the virtual pet in response to a user operation.

The word processor program 106 is a program for editing a document composed of characters or figures. The spreadsheet program 107 has a function of executing a predetermined operation on numerical values arranged in a table of a predetermined format, or drawing a graph corresponding to the arranged numerical values.

FIG. 9 is a diagram for explaining more detailed functions of the voice commander 102. The UI (user interface) processing unit 123 includes an application communication unit 121,
Engine communication unit 122, voice launcher control unit 124, user dictionary control unit 125, or recognition test processing unit 126
And a predetermined signal from the keyboard 4 or the stick-type pointing device 5 and the like, and the loudness of the voice input through the microphone 24 or the result of voice recognition is displayed in a predetermined window. To be displayed. When activating a predetermined program, the UI processing unit 123 displays a predetermined image on the LCD 21 based on data input from the application communication unit 121 or the audio launcher control unit 124.

The UI processing unit 123 changes the state of the UI processing unit 123 itself on the basis of a signal corresponding to the operation of the keyboard 4 or the stick type pointing device 5, and transmits predetermined data to the application communication unit 121.
Engine communication unit 122, voice launcher control unit 124, user dictionary control unit 125, or recognition test processing unit 126
To supply.

Further, the UI processing unit 123 controls the state of the still image photographing program 103, the still image browsing program 104, and the electronic pet program 105, and the user supplied from the voice recognition engine 101 via the engine communication unit 122. Based on data such as a predetermined text corresponding to a predetermined word spoken by the application communication unit 1
21 or the voice launcher control unit 124 determines whether to transmit a command or activate a predetermined program, and causes the application communication unit 121 or the voice launcher control unit 124 to transmit the command or activate a predetermined program. .

The application communication unit 121 includes a still image photographing program 103, a still image browsing program 104,
Alternatively, the electronic pet program 105 is activated, and communicates with the activated still image photographing program 103, still image browsing program 104, or electronic pet program 105, and the still image photographing program 103, the still image browsing program 104, or the electronic pet is activated. Data indicating each state is received from the program 105.

The application communication unit 121 includes a still image photographing program 103, a still image browsing program 104,
In addition to supplying data indicating the state of the electronic pet program 105 to the engine communication unit 122 or the UI processing unit 123, the engine communication unit 122 or the UI processing unit 123 outputs predetermined data corresponding to a predetermined word spoken by the user. It receives data such as text, data corresponding to a user's operation on the keyboard 4, and the like.

Also, the application communication unit 121
Still image shooting program 103, still image viewing program 1
04, the state of the electronic pet program 105, and the voice recognition engine 10 via the engine communication unit 122.
On the basis of data such as a predetermined text corresponding to a predetermined word spoken by the user supplied from 1, one of the still image photographing program 103, the still image browsing program 104, and the electronic pet program 105 is activated. ,
Alternatively, a predetermined command is supplied to any one of the still image photographing program 103, the still image browsing program 104, and the electronic pet program 105.

The still image photographing program 103, the still image browsing program 104, and the electronic pet program 105
Are not focused (when neither is active), the voice commander 102
Cannot execute a command for any one of the still image shooting program 103, the still image browsing program 104, and the electronic pet program 105.

A still image photographing program 103, a still image browsing program 104, and an electronic pet program 105
Are focused (when either is active), the voice commander 102
Is an active still image shooting program 103,
A command for either the still image browsing program 104 or the electronic pet program 105 can be executed.

Such a still image photographing program 103,
A command for a specific one of the still image browsing program 104 and the electronic pet program 105 is referred to as a local command.

The method by which the voice commander 102 specifies a program for transmitting a local command is not limited to the focus, but may refer to another state or data.

The engine communication unit 122 reads the recognized word data and the grammatical data from the recognized word / grammar database 115 based on a predetermined method, transmits the data to the voice recognition engine 101, and transmits the data to the voice recognition engine 101. Data such as a predetermined text corresponding to a predetermined word spoken by the user supplied from 101 is received.

The engine communication unit 122 is, for example, as shown in FIG.
Microsoft Speech API (trademark) (hereafter, SAPI
The speech recognition engine 10
1, the data of the recognized word and the data of the grammar stored in the recognized word / grammar database 115 are transmitted. In the data example shown in FIG. 10, the target of speech recognition is <Global>
And <SVCommand>, and <Global> is further (Chan
geWin), (VoiceCommand), where <SVCommand> is a command such as "Help" or "Previous", and <SendMail>
It is described that it also includes the mail command represented by. In the example of the data shown in FIG. 10, the code number of the command “help” is 102, and the word having the pronunciation “dad” is “daddy@test.company.co.jp”.
It is shown that the character string is related.

The speech recognition engine 101 converts the data received from the engine communication unit 122 into data of a predetermined method, and stores the data in the reading kana dictionary database 111 or the recognized word / grammar database 112 for the engine.
Based on the data stored in the reading kana dictionary database 111 or the engine recognition word / grammar database 112, a speech recognition process is executed.

The speech recognition engine 101 causes the engine communication unit 122 to correspond to a predetermined word spoken by the user.
A code number (eg, 102), a recognized word or sentence (eg, “Email Dad”), and a character string associated with the recognized word (eg, “daddy@test.compa”)
ny.co.jp ”).

For example, when the user inputs a voice of “mail to dad” into the microphone 24 and the voice recognition engine 101 correctly recognizes the voice, the voice recognition engine 101 outputs 7fffffff (hexadecimal), “ "E-mail to dad" and "daddy@test.company.co.jp" are transmitted to engine communication unit 122.

The engine communication unit 122 converts the received data based on the data received from the speech recognition engine 101 into an application communication unit 121, a UI processing unit 123, a speech launcher control unit 124, a user dictionary control unit 125, or a recognition test. It determines which of the processing units 126 to transmit, and based on the determination, converts the data received from the speech recognition engine 101 into a predetermined method, and selects the selected application communication unit 121, UI processing unit 123, The converted data is supplied to any one of the launcher control unit 124, the user dictionary control unit 125, and the recognition test processing unit 126.

The voice launcher control unit 124 displays an application program (e-mail program 54A, word processor program 10
6, or the setting relating to the activation of the spreadsheet program 107) is stored in the launcher setting database 113, and the setting relating to the speech recognition word or grammar stored in the recognition word / grammar database 115 is updated based on the setting.

When the data related to the launcher is received from the engine communication unit 122, the voice launcher control unit 124 executes the e-mail program 54 based on the setting related to the activation stored in the launcher setting database 113.
A, activates any one of the word processor program 106 and the spreadsheet program 107 and supplies a mail address and the like to the e-mail program 54A.

The voice commander 102 executes a command to activate any one of the electronic mail program 54A, the word processor program 106, and the spreadsheet program 107, regardless of the focus state (no matter which program is active). be able to.

Regardless of the focus state, the program can be always executed. For example, the electronic mail program 54A, the word processor program 10
6 or a command for activating one of the spreadsheet programs 107 is referred to as a global command.

The user dictionary control unit 125 displays a graphical user interface, stores the setting relating to the recognized voice input by the user in the dictionary setting database 114, and based on the setting,
The setting related to the word or grammar for speech recognition stored in the recognition word / grammar database 115 is updated.

When the user inputs a command to execute a test to the user dictionary control unit 125, the recognition test processing unit 126 displays a graphical user interface, and stores the graphical user interface in the dictionary setting database 114. The predetermined one word and the engine communication unit 122
And determines whether the word supplied from the voice recognition engine 101 matches the word indicating the result of the voice recognition, and displays the result of the determination.

When the user inputs a command to the user dictionary control unit 125 to execute a test, the recognition test processing unit 126 displays a graphical user interface and outputs a voice via the engine communication unit 122. It is determined whether or not the word indicating the result of the voice recognition supplied from the recognition engine 101 is stored in the dictionary setting database 114 and is included in one or more selected predetermined words. Is displayed.

When the voice commander 102 is activated, the UI
The processing unit 123 causes the LCD 21 to display an image indicating that the voice commander is running, displays the window of the voice commander 102 shown in FIG.

The voice commander window 161 includes a level gauge 162, a recognition result display section 163, a launcher setting button 164, a dictionary management button 165, and a help button 16.
6, a minimize button 167, a close button 168, a recognition state display section 169, and a voice input mode switching button 170.

The level gauge 162 is connected to the microphone 2
4 shows the level of the user's voice input via the microphone 4 (the amplitude of the signal output from the microphone 24). The recognition result display unit 163 displays a word or a sentence corresponding to the recognized voice supplied from the engine communication unit 122.
The launcher setting button 164 is used for the e-mail program 5
4A, the word processor program 106 or the spreadsheet program 107 is operated to make settings for activation.

The dictionary management button 165 is operated when the setting relating to the voice to be recognized is stored in the dictionary setting database 114. The help button 166 is operated when displaying online help on the LCD 21. The minimize button 167 allows the voice commander window 161 to be
The operation is performed when the icon is deleted from D21 and a predetermined icon is displayed on the task tray, for example. Close button 1
Reference numeral 68 is operated when the voice commander 102 is terminated.

The recognition state display section 169 displays the state of the speech recognition engine 101, whether a local command can be used (whether a predetermined program is active or not) and the like. Voice input mode switching button 170
Is operated when switching between the normal recognition mode and the normal recognition mode.

When the voice recognition engine 101 is activated, the voice commander window 161 as described above
Displayed in D21. When the close button 168 is clicked in such a state, the UI processing unit 123 ends the voice commander 102. When the user presses a key assigned to voice recognition (for example, a control key of the keyboard 4; hereinafter, referred to as a recognition key), the UI processing unit 123 transits to a state in which voice input is possible.

When transitioning to a state where voice input is possible, the UI
The processing unit 123 transmits the still image photographing program 103 and the still image browsing program 10 from the application communication unit 121.
4, and data indicating the active program among the electronic pet programs 105 is received, and the name of the active program is entered in the voice commander window 16.
1 is displayed on the recognition state display unit 169. When none of the still image photographing program 103, the still image browsing program 104, or the electronic pet program 105 is active, the UI processing unit 123 displays a message to that effect on the recognition status display unit 169 of the voice commander window 161 (for example, “Gl”).
obal Command ”).

In a state where voice input is possible, a signal corresponding to the voice input by the user from the microphone 24 is supplied to the voice recognition engine 101, and a signal corresponding to the level of the signal supplied to the voice recognition engine 101 is provided. Is supplied to the UI processing unit 123 via the engine communication unit 122. Also, the UI processing unit 123
Based on the data corresponding to the level of the signal corresponding to the voice, the level gauge 16 of the voice commander window 161 is used.
Update the display of 2.

Further, when the speech recognition engine 101 recognizes the speech, the UI processing unit 123 receives the data such as the recognized word or sentence from the speech recognition engine 101 and displays the data in the recognition result display unit 1 of the speech commander window 161.
63 displays the recognized word or sentence.

When the user releases the recognition key in a state in which the voice can be recognized, the UI processing unit 123 controls the application communication unit 121 or the voice launcher control unit 124.
A predetermined operation (for example, corresponding to the code number, the recognized word or sentence, and the character string data related to the recognized word, supplied from the voice recognition engine 101)
(E.g., activation of the e-mail program 54A).

At this time, the application communication unit 121
Starts one of the still image photographing program 103, the still image browsing program 104, and the electronic pet program 105 in response to a request from the UI processing unit 123, or executes the still image photographing program 103, the still image browsing program 104 Or electronic pet program 105
A predetermined command.

At this time, the voice launcher control unit 124
In response to the request from the UI processing unit 123, the e-mail program 54A, the word processor program 106,
Alternatively, one of the spreadsheet programs 107 is activated, or predetermined data (for example, a mail address) is supplied to the e-mail program 54A.

When the application communication unit 121 or the sound launcher control unit 124 completes a predetermined operation for a predetermined program, the application communication unit 1
21 or the voice launcher control unit 124
3 is notified to that effect, and the UI processing unit 123 causes the LCD 21 to display an image that allows the user to recognize the predetermined program to be operated according to the predetermined program to be operated.

An image for recognizing a predetermined program to be operated is displayed on the LCD 21, so that the user can know the result of voice recognition and the operation of the voice commander 102.

When the voice input mode switching button 170 is clicked, the UI processing section 123 transits to a state in which it is always in the recognition mode. When transitioning to the constant recognition mode, the UI processing unit 123
1 to a still image photographing program 103, a still image browsing program 104, and an electronic pet program 105,
Receiving data indicating the active program and recognizing the name of the active program
9 is displayed. Still image shooting program 103, still image viewing program 104, or electronic pet program 1
05 is not active, the UI processing unit 12
No. 3 causes the recognition status display section 169 of the voice commander window 161 to display the fact (for example, “Global Command”).

In the continuous recognition mode, when the voice recognition engine recognizes a predetermined voice irrespective of the operation of the recognition key, the voice commander 2 executes the still image photographing program 103, the still image browsing program 104, or the electronic pet program. 105, or sends a predetermined command to any one of the still image photographing program 103, the still image browsing program 104, and the electronic pet program 105, or the e-mail program 54A, the word processor program 106, or the spreadsheet. Activate any of the programs 107 or supply predetermined data to the e-mail program 54A.

When the voice input mode switching button 170 is clicked in the continuous recognition mode, the UI processing unit 123 shifts to the normal recognition mode.

When the dictionary management button 165 of the voice commander window 161 is clicked in the normal recognition mode, the UI processing unit 123 makes a transition to a state in which a dictionary is set, and the user dictionary control unit 125 sends a dictionary setting process. Request. In the state of setting a dictionary, the user dictionary control unit 125 causes the LCD 21 to display a dialog for setting a dictionary, and stores the dialog in the dictionary setting database 114 and the recognized word / grammar database 115 based on an operation on the dialog for setting a dictionary. Update the settings you have.

When a test button arranged in the dictionary setting dialog is clicked in a state where a dictionary is set, the UI processing unit 123 makes a transition to a state in which a speech recognition test is executed, and the recognition test processing unit 126 Request processing of the voice recognition test. The recognition test processing unit 126
A dialog for the voice recognition test is displayed on the LCD 21, and the voice recognition engine 101 is transmitted via the engine communication unit 122.
Performs a speech recognition test for determining whether or not the word whose speech has been recognized supplied from is matched with a word registered in the dictionary setting database 114, and displays the result.

Alternatively, the recognition test processing unit 126 causes the LCD 21 to display a dialog of the speech recognition test, and the recognized word supplied from the speech recognition engine 101 via the engine communication unit 122 is used by the dictionary setting database 11
Then, a speech recognition test for determining whether the word is included in the words registered in No. 4 is executed, and the result is displayed.

In the state where the voice recognition test is executed,
When a test button arranged in the dialog for the voice recognition test is clicked, the UI processing unit 123 transits to a state where a dictionary is set. When the close button arranged in the dictionary setting dialog is clicked in the state where the dictionary is set, the UI processing unit 123 transits to the normal recognition mode.

When the launcher setting button 164 of the voice commander window 161 is clicked in the normal recognition mode, the UI processing unit 123 causes the voice launcher control unit 1
24 e-mail program 54A, word processor program 106, or spreadsheet program 107.
24 is requested to perform the setting process of the program activation.

In a state in which the setting of starting is performed, the voice launcher control unit 124 displays a dialog for launcher setting in the LC.
D21 is displayed, and the setting stored in the launcher setting database 113 is updated based on the operation on the launcher setting dialog.

Next, the LCD 2 of the personal computer 1
The screen displayed in No. 1 will be described. FIG. 12 is a diagram illustrating a screen displayed on the LCD 21 when the voice commander 102, the voice recognition engine 101, and the electronic pet program 105 are running.

At a predetermined position on the screen of the LCD 21, an icon 181 corresponding to the electronic mail program 54A and an icon 18 corresponding to the word processor program 106 are displayed.
2. Icon 18 corresponding to spreadsheet program 107
3. Voice commander window 161 and electronic pet window 1 displayed by electronic pet program 105
91 are arranged.

When the stick type pointing device 5 or the like is operated to select the icon 181 and execute a start command (an operation such as selecting from a menu or the like not shown), the e-mail program 54A is started. . When the icon 182 is selected and the start command is executed, the word processor program 106 is started. When the icon 183 is selected and a start command is executed, the spreadsheet program 107 is started.

The electronic pet window 191 displays the electronic pet inhabiting the virtual space.
And a plurality of buttons 201 to 206. The close button 201 is used for the electronic pet program 10
5 is operated when the user wants to end
Is operated when the display of the electronic pet window 191 is to be enlarged, and the minimize button 203 is operated when the electronic pet window 191 is to be deleted from the LCD 21 and stored in the task bar.

The scale instruction button 204 is operated when giving an instruction to a robot (not shown), and the dictionary management button 205 is operated when it is desired to register a new name of the virtual pet in the dictionary. Help button 206
Is operated when you want to check what you do not know.

In the state shown in FIG. 12, the user
When the minimize button 167 of the voice commander window 161 is operated, as a result of the operation, the task tray 211
An icon corresponding to the voice commander 102 is shown in FIG.
It is displayed as shown in FIG. In FIG. 13, among a plurality of icons displayed on the task tray 211, an icon 221 is an icon corresponding to the voice commander 102.

When the user moves the cursor 231 on this icon 211 and performs a predetermined operation such as clicking, a menu is displayed as shown in FIG. The menu includes a “normal display” which is operated when the voice commander window 161 is to be normally displayed (when it is desired to return to the display as shown in FIG. 12), and which is operated when switching between the normal recognition mode and the normal recognition mode. "Input mode switching" and "end" operated when terminating the voice commander 102 are displayed.

In the display state as shown in FIG. 13, that is, when the voice commander window 161 is minimized and the icon 221 is displayed on the task tray 211, the user performs voice recognition. Therefore, when the recognition key is operated, a small display window 241 is displayed as shown in FIG. The small display window 241 is displayed near the activated (activated) voice recognition compatible application when the recognition key is operated. For example, FIG.
In the case of 5, the electronic pet program 105 is activated as a voice recognition compatible application, and is displayed so as not to overlap the upper left of the electronic pet window 191 of the electronic pet program 105.

The electronic pet window 191 itself has an LCD
There is a case where it is cut off because it is at the end of 21. In such a case, as shown in FIG. 16, the small display window 241 is displayed so as to overlap the electronic pet window 191. The position of the small display window 241 with respect to the application may be anywhere, such as the upper right, lower right, or lower left, and is not limited to the upper left.

When an application corresponding to voice recognition is activated and the application is in an active state, when the recognition key is operated as described above, a small size is placed near the application. Although the display window 241 is displayed, when an application corresponding to voice recognition is not activated, or when it is activated but is not in an active state, for example, as shown in FIG. , A small display window 241 is displayed at the lower center of the screen of the LCD 21. Of course, the display position is not limited to the lower side of the center.

The round part on the left side of the small display window 241 is a part for performing the same display as the level gauge 162 (FIG. 11), and the elongated part on the right side is the recognition state display unit 1.
This is a portion for displaying the same as 69 (displaying the result of voice recognition).

FIG. 18 shows a state in which two applications corresponding to voice recognition are activated.
FIG. 18 illustrates a state in which the electronic pet program 105 and the still image browsing program 104 have been activated as applications supporting voice recognition. In such a state, when the user operates the recognition key while the still image browsing program 104 is in an active state, a small display window 241 is displayed on the upper left of the active still image browsing window 251. Is displayed.

As described above, to which application the result of voice recognition is to be sent, in other words, to which application a voice command is to be issued, is displayed in the small display window 241 near the window corresponding to the application. Is displayed, the user can be recognized.

Also, for example, the small display window 241 is positioned near the still image browsing window 251 even though the user operates the recognition key to give a command to the electronic pet program 105 by voice. When the electronic pet program 105 is displayed or displayed at the lower center of the screen, the user can determine that the electronic pet program 105 is not in an active state, and based on the determination, activates the electronic pet program 105. Then, the user can operate the recognition key again to issue a command to the electronic pet program 105. That is, it is possible to prevent a command from being issued to an incorrect application.

Further, such a small display window 2
41, the voice commander window 161
Can be reduced in the time required for the drawing (processing capability) as compared with the case where is normally displayed.

By the way, in order to perform voice recognition, it is necessary to speak in accordance with a preset command format. When executing a process by voice recognition, the user sometimes does not know what command format should be used in order to execute a desired process. In such a case, the user speaks “command list” to display a help screen for the command on the LCD.
21 can be displayed.

FIG. 19 shows the result of the user saying “command list”. This shows a state where the command is recognized as “command list” and displayed on the recognition result display section 163 of the voice commander window 161. Of course, since the voice command window 161 is instructed to minimize the display,
When the small display window 241 is displayed, "command list" is displayed in the small display window 241. In this way, by recognizing the word “command list” spoken by the user, a help screen relating to voice commands is displayed on the LCD 21 as shown in FIG.

As described above, when it is difficult to operate a command related to voice recognition, the help screen can be displayed by voice so that the user can
It is not necessary to perform an operation of displaying a help screen by operating a mouse or the like, and the operation can be consistently performed by voice, thereby improving usability.

In speech recognition, as the number of uses increases, learning is performed, and the recognition rate increases. Therefore, the user sometimes wants to know information on how many times the voice commander 102 has been used. In such a case, for example,
By saying the word "how many times have you used?", The number of times the voice commander 102 has been used can be known.
Specifically, when the user utters “How many times have you used?”, The recognition result is displayed on the recognition result display section 163 of the voice commander window 161 as shown in FIG. Is done. Then, as shown in FIG. 21 (B), the number of times of use is displayed on the recognition state display unit 169 (in the case of the display example of FIG. 21 (B), the number is eight). This display is turned off after being displayed for a predetermined time.

Next, the processing of command transmission by voice of the CPU 52 executing the voice recognition engine 101 and the voice commander 102 will be described with reference to the flowchart of FIG. Here, the description will be given on the assumption that the voice commander window 161 has been instructed to minimize the display and that one or more applications corresponding to voice recognition have been activated.

In step S1, the voice commander 10
Step 2 determines whether or not the continuous input mode is selected. If it is determined that the continuous input mode is not selected, the process proceeds to step S2 to determine whether or not the recognition key is pressed. If it is determined in step S2 that the recognition key has not been pressed, the process returns to step S2, and the voice commander 102 repeats the process of determining whether the recognition key has been pressed until the recognition key is pressed.

If it is determined in step S2 that the recognition key has been pressed, the process proceeds to step S3, in which the small display window 241 is placed in the vicinity of an active application corresponding to voice recognition, here, It is displayed at the upper left (as shown in FIG. 15). Then, in step S4, the voice commander 102
It causes the speech recognition engine 101 to execute speech recognition processing. In step S5, the voice commander 102 receives the result of the voice recognition processing from the voice recognition engine 101.

At step S6, the voice commander 10
2 displays the result of the speech recognition processing received from the speech recognition engine 101 on the small display window 241. In step S7, the voice commander 102 determines whether or not the recognition key has been released. If it is determined that the recognition key has been released, the process proceeds to step S8, where the displayed small display window 241 is erased from the LCD 21. In step S9, a command is transmitted, and the process ends.

If it is determined in step S7 that the recognition key has not been released, the flow advances to step S10, where the voice commander 102 sets the release key (for example, the shift key).
It is determined whether or not is pressed. If it is determined in step S10 that the release key has been pressed, the process proceeds to step S11, in which the voice commander 102 clears the result of voice recognition, returns to step S4, and repeats the voice recognition process. If it is determined in step S10 that the release key has not been pressed, step S11 is skipped, the process returns to step S4, and the voice recognition process is repeated.

On the other hand, if it is determined in step S1 that the constant input mode has been selected, the process proceeds to step S1.
Proceeding to 2, the small display window 241 is displayed at the upper left of the application corresponding to the active voice recognition. Then, in step S13, the voice commander 102 makes the voice recognition engine 101
To execute voice recognition processing. In step S14, the voice commander 102 receives the result of the voice recognition process from the voice recognition engine 101.

In step S15, the voice commander 1
02 displays the result of the speech recognition process received from the speech recognition engine 101 on the small display window 241. In step S16, the voice commander 102
After transmitting the command, the process returns to step S1 to repeat the process of transmitting or starting the command.

As described above, by operating the recognition key, even when the voice commander window 161 is displayed in a minimized state, the user confirms the result of voice recognition in the small display window 241 and checks the command. Transmission can be performed, and execution of processing due to incorrect recognition can be prevented. Further, as described above, the user can easily play with the electronic pet or browse the still image without operating the keyboard 4 or the like.

The above-described series of processing can be executed by hardware, but can also be executed by software. When a series of processing is executed by software, a program constituting the software can execute various functions by installing a computer built into dedicated hardware or installing various programs. It is installed from a program storage medium to a possible general-purpose personal computer or the like.

As shown in FIG. 7, a program storage medium for storing a program installed in a computer and made executable by the computer includes a magnetic disk 95 (including a floppy disk), an optical disk 96 (CD-ROM ( Compact Disc-Read Only Memory), DVD (D
digital Versatile Disc), magneto-optical disc 97
(Including MD (Mini-Disc)) or semiconductor memory 9
8 or a ROM 69 for temporarily or permanently storing a program, a hard disk 56, or the like. The storage of the program in the program storage medium can be performed by a router,
This is performed using a wired or wireless communication medium such as a local area network, the Internet 92, or a digital satellite broadcast (not shown) via an interface such as the modem 50.

In this specification, the step of describing a program stored in a program storage medium is not limited to processing performed in chronological order according to the described order, but is not necessarily performed in chronological order. , And also includes processes executed in parallel or individually.

In this specification, the system is
It represents the entire device composed of a plurality of devices.

[0126]

As described above, the information processing apparatus according to the first aspect, the information processing method according to the third aspect, and the fourth aspect.
According to the program storage medium described in the above, a program for executing a predetermined process corresponding to the result of voice recognition is started, and when it is determined that it is in an active state, voice recognition is instructed The display is controlled so that the first window indicating the present state is displayed near or overlapping with the second window corresponding to the program, and a predetermined process is executed in response to the result of voice recognition. If it is determined that the program has not been started or has been started but is not active, the first
Since the display is controlled so that the window is displayed at a predetermined position, the user can confirm the state of the voice recognition, and thus can instruct an erroneous process. Can be prevented.

[Brief description of the drawings]

FIG. 1 is an external perspective view of a personal computer 1. FIG.

FIG. 2 is a plan view of the personal computer 1.

FIG. 3 is a left side view of the personal computer 1.

FIG. 4 is a right side view of the personal computer 1.

FIG. 5 is a front view of the personal computer 1.

FIG. 6 is a bottom view of the personal computer 1.

FIG. 7 is a block diagram showing a configuration of the personal computer 1.

FIG. 8 is a diagram showing functional blocks of the personal computer 1.

FIG. 9 is a diagram showing functional blocks of the personal computer 1.

FIG. 10 is a diagram illustrating SAPI.

FIG. 11 is a diagram showing a voice commander window 161.

FIG. 12 is a diagram illustrating a screen displayed on the LCD 21.

FIG. 13 is a diagram illustrating a screen displayed on the LCD 21.

FIG. 14 is a diagram illustrating a screen displayed on the LCD 21.

FIG. 15 is a diagram illustrating a screen displayed on the LCD 21.

FIG. 16 is a diagram illustrating a screen displayed on the LCD 21.

FIG. 17 is a diagram illustrating a screen displayed on the LCD 21.

FIG. 18 is a diagram illustrating a screen displayed on the LCD 21.

FIG. 19 is a diagram illustrating a screen displayed on the LCD 21.

FIG. 20 is a diagram illustrating a screen displayed on the LCD 21.

FIG. 21 is a diagram illustrating a screen displayed on the LCD 21.

FIG. 22 is a flowchart illustrating processing related to speech recognition.

[Explanation of symbols]

1 personal computer, 4 keyboard, 1
0 shutter button, 21 LCD, 23 CCD video camera, 24 microphone, 52 CPU, 59
ROM, 54 RAM, 84A E-mail program, 86 Front switch, 88 drive, 9
2 Internet, 95 Magnetic disk, 96
Optical disk, 97 magneto-optical disk, 98 semiconductor memory, 101 voice recognition engine, 102 voice commander, 103 still image photographing program, 104
Still image browsing program, 111 reading kana dictionary database, 112 recognition word / grammar database for engine, 113 launcher setting database, 114
Dictionary setting database, 115 recognized word database, 121 application communication unit, 122
Engine communication unit, 123 UI processing unit, 124 voice launcher control unit, 125 user dictionary control unit, 12
6 Recognition test processing unit, 241 Small display window

──────────────────────────────────────────────────の Continued on the front page (51) Int.Cl. ⁷ Identification code FI Theme coat ゛ (Reference) G10L 15/00 G10L 3/00 551P 15/28 561C 15/22 571H (72) Inventor Shuji Yonekura Shinagawa, Tokyo 6-7-35 Kita-Shinagawa-ku, Sony Corporation (72) Takashi Sasai Inventor 6-35, Kita-Shinagawa, Shinagawa-ku, Tokyo Sony Corporation F-term (reference) 5B076 AB17 5D015 KK03 LL02 LL05 5E501 AA02 AC37 CA03 CB15 EA21 EB05 FA06 FA23 FA43 FB22 9A001 DD11 HH17 HH34

Claims (4)

[Claims] A first determining means for determining whether or not a state of recognizing a voice is instructed; and if the first determining means determines that the state of recognizing the voice is instructed, A second determining means for determining whether or not a program for executing a predetermined process according to a result of the voice recognition is activated and in an active state; and When it is determined that the program for executing the predetermined process corresponding to the result is activated and is in the active state, the first window indicating the state where the voice recognition is instructed is added to the program. A first controlling display so as to be displayed near or overlapping with the corresponding second window;
The display control unit and the second determination unit determine that the program for executing the predetermined process in response to the result of the voice recognition is not activated, or is activated but not active. And a second display control means for controlling the display so that the first window is displayed at a predetermined position. 2. A third display for controlling the display so that the list of commands is displayed, when the result of the speech recognition is an instruction to display a list of commands for executing a predetermined process. The information processing apparatus according to claim 1, further comprising a control unit. 3. A first judging step of judging whether or not a state of recognizing a voice has been instructed, and it has been determined in the processing of the first judging step that a state of recognizing a voice has been instructed. A second determining step of determining whether or not a program for executing a predetermined process according to a result of the voice recognition is activated and in an active state; and a process of the second determining step When a program that executes a predetermined process corresponding to the result of voice recognition is started, and it is determined that it is in an active state,
A first display control step of controlling display so that a first window indicating a state in which voice recognition is instructed is displayed near or overlapping with a second window corresponding to the program; If it is determined in the processing of the second determination step that the program for executing the predetermined processing corresponding to the result of the voice recognition has not been activated, or that it has been activated but is not in an active state, A second display control step of controlling the display so that the first window is displayed at a predetermined position. 4. A first judging step of judging whether or not a state of recognizing a voice has been instructed, and it has been determined in the processing of the first judging step that a state of recognizing a voice has been instructed. In the case, a second determination step of determining whether or not a program for voice recognition is activated and is in an active state; and a predetermined processing corresponding to a result of voice recognition in the processing of the second determination step. If it is determined that the program that executes the processing of the above is activated and is in the active state,
A first display control step of controlling display so that a first window indicating a state in which voice recognition is instructed is displayed near or overlapping with a second window corresponding to the program; If it is determined in the processing of the second determination step that the program for executing the predetermined processing corresponding to the result of the voice recognition has not been activated, or that it has been activated but is not in an active state, And a second display control step of controlling display so that the first window is displayed at a predetermined position. A program storage medium storing a computer-executable program. .