JPH04338817A - Electronic equipment controller - Google Patents

Abstract

PURPOSE:To obtain a controller which can accurately analyzes the purpose of the instruction given from a user and then control an electronic equipment in place of the user who is not accustomed to operate the electronic equipment. CONSTITUTION:The voices of a user 50 using an electronic equipment like a VTR 71, etc., are recognized by a speech recognizing part 62. A conversation comprehending part 63 infers or comprehends the instructions of the user 50 based on the recognizing result of the part 62. Based on this comprehending result, a control part 66 controls the electronic equipment. At the same time, a message preparation part 67 produces a message to be given for the user 50 based on the inferring result of the part 63.

Description

[Detailed description of the invention]

0001

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a control device for electronic equipment that controls various electronic equipment in response to instructions from a user.

0002

2. Description of the Related Art In recent years, various electronic devices have become more sophisticated due to the spread of so-called microprocessors, and as a result, the operation of these electronic devices tends to become more complex. Under these circumstances, there are users who are unable to fully utilize complex electronic devices.
is increasing.

The following are possible reasons why electronic devices become difficult for so-called "electronic device weak" users who are not good at operating such electronic devices. First, operating electronic devices is complicated and difficult to learn. For example, in recent years, electronic devices have increased the number of functions, and accordingly, the number of operations has also increased. In other words, in order to realize such a large number of operations in these electronic devices, it is necessary to increase the number of buttons and switches according to the number of operations, or to realize various operations by combining fewer buttons. I decided to make it
In either case, operating electronic devices becomes complicated. It is extremely difficult for users who are not familiar with electronic devices to remember all of these complicated operations. Furthermore, it is difficult for the user to associate his/her own intention (what the user is thinking) with various operations of the electronic device. That is,
For example, there are users who are unable to operate the electronic device even after reading the instruction manual for the device, and in this case, there is a gap between the expression of the user's personal intentions and the expression of the function description in the instruction manual. Because of this, users may not be able to connect the two.

0004

[Problems to be Solved by the Invention] In order to solve these problems, the above-mentioned electronic devices, such as document creation devices (so-called word processors), provide explanations of operation methods on the screen of the document creation device. , or various attempts such as sending messages to users by voice (Va
lerie L. Frankel & Osman
Balci, “An on-line assistant
ance system for the simulator
ion model development env
Ironment”, Int. J. Man-Ma
Chine Studies, 31, pp. 699-
716, 1989, Noriyuki Ito et al., “Effects of voice messages in interactive computer systems for general users,” Proceedings of the 3rd Human Interface Symposium, pp. 319-324, 1987). These can be said to be attempts to incorporate part of the instruction manual into electronic devices. Such an approach is effective for users who cannot use electronic devices simply because they cannot remember the operating procedures.

[0005] However, the above-mentioned attempts are not necessarily effective even for users who cannot fully use electronic equipment even after reading the instruction manual. Furthermore, the transmission of information to users from instruction manuals and so-called help screens is fixed and one-sided; for example, it is difficult to change the wording depending on the user's level of understanding, or to respond to detailed questions from the user. There is little flexibility.

[0006] In addition, for relatively simple tasks that do not require much judgment on the part of the user, electronic equipment can automatically perform fine adjustment work under various conditions that has previously been performed by the user using so-called fuzzy control. (Kaoru Hirota, “Current status and prospects of fuzzy information processing applications”, Information Processing Society of Japan, vol. 30, no. 8)
, 1989, pp. 913-921). according to this,
This has the effect of reducing the effort required for the user to perform operations and preventing erroneous operations. However, this method is difficult to apply to complex tasks that require a lot of input information from the user, such as the recording reservation of a video tape recorder (VTR), which will be described later.

Furthermore, attempts have been made to promote understanding of unknown phenomena by applying so-called metaphors to the interface between the user and electronic equipment, that is, the man-machine interface.
Brenda Laurel, “Interfa
ce Agent: Metaphors with
Character” The Art of Hum
an-Computer Interface Des
ign, pp. 355-366, 1990, Ken
tL. Norman & John P. Chi
n, ”The menu metaphor: fo
"oddfor thou", Behavior
and Information Technology
gy, vol. 8, no. 2, pp. 125-134
, 1989, Stuart K. Card & D.
Austin Henderson, Jr. , ”
Catalogs: A metap-horfo
r computer application
Human-Computer I
interradtion-Interact'87, p.
p. 959-964, 1987), an attempt to change interface functions to suit the user's level (
Susumu Yoshimura et al., “About user-friendly assistance,” Proceedings of the 2nd Human Interface Symposium, pp. 39-42, 1986) (Kyoji Kawagoe et al., “Intelligent Dialogue System with Adaptive Dialogue Interface and its Applications”, Proceedings of the 2nd Human Interface Symposium, pp. 25-32, 1986)
, Hideyuki Takagi, “Subjective evaluation of voice input word processor”
, Proceedings of the 2nd 2nd Human Interface Symposium, pp. 351-360, 1986). These are attempts to change the way the functions of electronic devices are presented in a way that is easier for users to understand, but they do not actively elicit correct understanding from users.

[0008]The present invention was proposed in view of the above-mentioned circumstances, and provides a control method for electronic equipment that can control the operation of electronic equipment in place of a user who is unaccustomed to operating electronic equipment. The purpose is to provide a device.

[0009]

[Means for Solving the Problems] A control device for an electronic device according to the present invention has been proposed to achieve the above-mentioned object. a voice recognition unit that recognizes voice from a person; and control information or inference results that are understood by understanding or inferring the recognition results from the voice recognition unit in relation to at least control information for controlling electronic equipment; a dialogue understanding unit that outputs information; a message generation unit that receives the inference result information from the dialogue understanding unit and generates at least message information for asking questions or asking the user for confirmation; and a control section to which the control information is input and controls the electronic device.

[0010] Here, the control device for the electronic device prepares a control information table for controlling the electronic device and a user command information table (vocabulary information table), and the voice recognition unit The command (voice) from the user to control the electronic device is analyzed based on the vocabulary information table, the recognition result from the voice recognition unit is compared with the control information table, and the dialog understanding unit compares the recognition result from the voice recognition unit with the control information table. As a result of the comparison, if the command from the user and the control information do not match, control information that is determined to be optimal is selected, and an inquiry signal is sent to the user requesting permission to output the selected control information. is generated in the message generator.

[0011] Furthermore, as the speech recognition unit, for example, a so-called NAT type speech recognition device is used to recognize words in a vocabulary area registered in advance. In addition, the dialogue understanding unit uses general common sense and knowledge of the user's habits to analyze the context of the dialogue while interpolating and inferring missing information in order to understand the user's intentions, and uses it according to the dialogue situation. The message generation unit generates a message for asking a question or requesting confirmation from the person. The message signal from this message generation section is transmitted to the user as audio or video. For example, the video may be a visual representation of an animated character on a television screen. Furthermore, the dialogue is performed using a natural language synthesized using, for example, regular speech synthesis as the voice.

0012

According to the present invention, the electronic device is controlled in accordance with the control information understood based on the recognition result by recognizing the voice from the user. At this time, a message corresponding to the inference result information based on the recognition result is transmitted to the user, the user answers this message, and by further reasoning or understanding the answer, the user's command is The intention of the user can be understood accurately, and therefore the electronic equipment can be controlled accurately.

[0013]

Embodiments Hereinafter, embodiments to which the present invention is applied will be described with reference to the drawings. As shown in FIG. 1, the control device for electronic devices of this embodiment includes a control device 60 that controls electronic devices such as a video tape recorder (VTR) 71, a tape recorder 72, and a television receiver (TV) 73.
The relationship between a voice recognition unit 62 that recognizes voice from a user (user 50) using an electronic device based on a preset vocabulary information table, and control information for controlling at least the electronic device. Below the voice recognition section 62
a dialogue understanding unit 63 that understands or infers the recognition results from the user and outputs the understood control information or inference result information; The message generation section 67 generates at least message information to be obtained, and the electronic device control section 66 receives the control information from the dialogue understanding section 63 and controls the electronic device.

That is, the description of the operation of electronic equipment such as the VTR 71 can be expressed using a limited vocabulary area. Therefore, in this example, an ungrammatical input sentence (
A robust semantic analysis system that can also understand (natural language input by voice input from the user 50) (Seiichi Misawa et al., “Interactive intelligent interface and its application to VTR recording reservation”,
Proceedings of the 2nd Human Interface Symposium, pp. 207-264, 1991, Seiichi Misawa et al., “Interactive intelligent interface driven by semantic analysis”
, Proceedings of the 40th National Conference of the Information Processing Society of Japan (1),
pp. 448-449, 1990).

The voice recognition unit 62 configured as shown in FIG.
Shinichi Tamura et al., “NAT method speaker-independent word recognition”, Proceedings of the Acoustical Society of Japan, pp. 21-22, 1985) to recognize words on a pre-registered vocabulary area (command information table corresponding to commands from the user).

[0016] Furthermore, the dialog understanding unit 63 uses general common sense and knowledge of the habits of the user 50 (a table for analyzing commands from the user) to interpolate and infer missing information in order to understand the intention of the user 50. while understanding the context of the dialogue.

The message generating section 67 generates a message signal depending on the situation (for example, asking the user 50 a question or requesting confirmation depending on the dialog situation) according to the information (inference result information) transmitted from the dialog understanding section 63. A message signal for the purpose of the character is generated and sent to the voice synthesis section 65 and the character display section 68 for actual display. The message signal from the message generation section 67 is transmitted to the user as audio or video. For example, as a video, graphic data of an animation character AC, which will be described later, is combined on the character display section 68 based on the above-mentioned message signal, and the animation character AC is given movements and expressions according to the dialogue scene. The images are visualized and shown on the John screen (monitor 69). In addition, as for the voice, a natural language voice synthesized by the voice synthesis unit 65 using, for example, a so-called regular voice synthesis method is emitted from the speaker 64 so as to perform a dialogue with the user.

Furthermore, the electronic device control section 66 controls the actual electronic device according to the intention of the command of the user 50 understood through the interaction with the user 50, that is, based on the understood control information from the interaction understanding section. (For example, VTR71
etc.).

By the way, in order to create an environment where users and electronic devices can easily communicate,
The following two points are considered necessary. That is, firstly, electronic devices must have rich communication capabilities, and secondly, they must have the intention to understand the user's intentions. Note that the above-mentioned communication refers not only to dialogue, but also to the exchange of ideas through various means, including facial expressions and gestures.

[0020] No matter what method is used, if a device can be made to appear to exchange various information with the user with the intention of understanding the user's intentions, it is possible to It is possible to bring communication between humans and machines closer to that between humans. For example, let us say that an intelligent being that behaves in this way is called a machine will. In an interface environment where machine will exists, users can easily convey their intentions to the machine without using complicated operations. This means making machines look like humans. However, humans usually do not feel that machines are human, and in fact, they tend to dislike feeling like machines.

Therefore, in the embodiment of the present invention, as a means to realize such a conscious presence on the side of the electronic device, a person who stands between the user and the electronic device and mediates communication is used as a metaphor. An agent (control device 60 described above) is introduced into the human interface. In other words, an agent is placed between the user and the electronic device that interacts with the user with the intention of understanding the user's intentions and operates the electronic device on behalf of the user, like a human.

[0022] In this embodiment, as a metaphor for this, a person skilled in operating the electronic equipment is used who can operate the electronic equipment according to the user's intention without fail. Such an agent has the knowledge and ability to operate electronic devices as well as the intention to realize the user's intention, so the user can easily convey his or her intention and have the user perform the operation on his/her behalf. By creating such an interface environment, it becomes easier for users who have traditionally found it difficult to operate electronic devices to use the devices. The outline of this interface is the configuration of the present embodiment shown in FIG. 1 described above, and FIG. 2 shows a further simplified version of the configuration of FIG. 1.

In FIG. 2, there are the following three functions of the agent of the control device 60, which is a metaphor for a person skilled in operating the electronic device 70.

That is, the control device 60 (agent) has, as a first function, knowledge regarding the operation of the electronic device 70 and knowledge regarding the user, and uses this knowledge to correctly understand the intention of the user 50. . Also, the second
As a function, if the electronic device 70 cannot be operated due to insufficient input from the user 50, the user 50 is asked for necessary information. Furthermore, as a third function, the electronic device 70 can be operated accurately.

Here, as the first function, by having a certain degree of general knowledge, common sense, and knowledge regarding the operation of the electronic device 70, the user can easily understand from ambiguous expressions, expressions with partially omitted information, etc. 50 interpolates and infers information that was not clearly expressed. Furthermore, by having knowledge about the user's preferences and habits, it is possible to infer the user's 50 intention even from little information.

[0026] Also, as the second function, the first function is as follows.
If there is any information that you cannot understand using the function, please
Inquire about. Even when the user 50 does not know how to input, or seems confused, such as not knowing what to input, the user 50 is encouraged to voluntarily work on the user to obtain information.

Furthermore, the third function is to accurately operate the electronic device 70 as intended by the user 50 based on the information obtained as a result of the execution of the first and second functions.

[0028] In addition to the above-mentioned abilities, a skilled human operator has various characteristics that are effective in facilitating communication, and these are also included in this implementation. This is achieved in the example. For example, the agent is given a specific appearance (animation character AC) similar to a human being, as will be described later. For example, by presenting this animated character AC in front of the user 50, the user 50 is made to feel that he or she is communicating with this animated character AC, thereby inducing dialogue.

[0029] In other words, it is possible to predict the reaction of the other person to a certain extent when communicating because it is assumed that humans have emotions equally and that at least the fundamental part is common to all people. It is. Without this, you won't know how to interact with the other person, and you won't be able to communicate smoothly. In the embodiment of the present invention, communication is facilitated by having the agent express emotions similar to those of humans. Furthermore, as a means of expressing emotions, the animated character AC is given facial expressions and gestures, as will be described later. Normally, emotions can be expressed only by voice or by talking in text, but in this embodiment, the expression is visually expressed by the animated character AC, which is more effective.

Furthermore, each human being is a being with consistent accuracy, and it can be said that this consistency is a prerequisite for making it possible to accumulate information about the other person. Therefore, the more information you have about the other party, the more efficient your communication will be. Therefore, in this embodiment, the agent's emotional expression is made consistent, and is expressed with a certain degree of precision, such as being angry or patient.

As described above, in this embodiment, by providing the agent of the electronic device control device 60 with the above function, it can behave in the same way as an electronic device expert who has the intention of helping the user 50. can be done.

[0032] Furthermore, the electronic device control device 60 of this embodiment
, the basic flow of the interface is to convey the intention in words and perform the operation desired by the user 50 on the electronic device 70.
The flow of executing the program can be understood by all users with simple instructions. Furthermore, it is easy to use because it can be used in everyday language with fewer restrictions on the words that can be used, making it extremely convenient for people who cannot operate the device themselves or who are using this interface for the first time. In addition, the user has a favorable impression of the visual character (animated character AC) on the screen because it makes movements that express human emotions, and the user feels a sense of familiarity with the device. This also has the effect of reducing psychological negative reactions to communicating with machines.

That is, in the device of this embodiment, an agent that operates the device on behalf of the user is introduced into the human interface in order to make the electronic device, which is complicated to operate, easy to use even for users who are weak with machines. This agent is a metaphor for an expert in equipment operation, and in addition to knowledge of equipment operation, it also understands the user's intention and has the will to carry it out, so the interaction between the user and the equipment is improved. It is possible to facilitate communication. Therefore, even users who are not accustomed to operating electronic devices can easily use the devices.

[0034] Here, as the electronic device 70, for example, a home VTR can be said to be a multi-functional device that is difficult to operate. In particular, the operation for reserving recording of a program is complicated and many users cannot master it. Recording reservation is a function that cannot be processed unless the user specifies a lot of information in order to specify the program to be reserved, and the reason for this is that the conventional button operation method makes the operation complicated. The applicant has previously published Japanese Patent Application No. 3-13758, Japanese Patent Application No. 3-13759, Japanese Patent Application No. -13760,
A control device for electronic equipment is proposed as described in the specification and drawings of Japanese Patent Application No. 3-13761.

A specific example of a control device according to this embodiment will be described below. In this control device, the interface of the home VTR mentioned above is taken as an example. Of course, the control device of the present invention is not limited to the embodiments described above or the specific examples below, and can be applied to various other electronic devices such as a television receiver, a tape recorder, or a document creation device.

The control device 60 that performs the interface between the VTR and the user in this specific example includes an agent that has knowledge of VTR operations such as recording reservations and the ability to actually perform them. It has been introduced. As this agent, an animated character AC is visualized on a television screen. The user 50 interacts with this animated character (agent) AC using natural language using voice.

That is, in the specific device of this embodiment shown in FIG. 3, as a voice input means for inputting commands and various information for specifying and controlling the operation mode by voice,
For example, a handset 10 having the same shape as a so-called telephone handset is provided. Transmitter 11 of this handset 10
is an audio system that converts input audio into an electrical signal and outputs it.
An electrical conversion element is provided, and a switch (press talk switch) 12 is provided near the transmitting section 11 for specifying switching of the audio input state and, for example, adding a break between inputs. The output signal from the transmitter 11 of the handset 10 is sent to the voice recognition circuit 13 and subjected to signal processing, thereby recognizing the above-mentioned commands and various information. switch 1
The output signal from 2 is sent to the switch state detection section 14, and the on/off state is detected. These voice recognition circuit 13 and switch state detection section 14 are connected to a control circuit 15. In addition, the control circuit 15 includes an animation character generation circuit 16 that outputs a video signal of an animation character who is a speaker of a message, and an animation character generation circuit 16 that outputs a video signal of an animation character who is a speaker of a message, and a method that generates an audio signal of the message according to the input of the message signal, for example, by a method of so-called regular speech synthesis. A voice synthesis circuit 19 for synthesizing using the VTR 40 is connected to a VTR controller 18 for specifying and controlling the operation mode of the VTR 40. The video signal from the character generation circuit 16 is sent to a superimposer 17, superimposed on the video signal from the VTR 40, and sent to a cathode ray tube (CRT) display device 30 as an image display means. The audio signal from the audio synthesis circuit 19 is sent to a speaker 20 as audio output means and output as audio. The speaker 20 and CRT display device 30 may be configured together as a television receiver, for example.

The control circuit 15 generates an operation mode designation control signal for designating and controlling the operation mode of the VTR 40 and the operation of the animation character AC of the animation character generation circuit 16 in accordance with at least the output signal from the voice recognition circuit 13. It outputs an operation control signal to be controlled and a message signal instructing the message voice to be synthesized by the voice synthesis circuit 19, and includes a CPU such as a microprocessor. This control circuit 15 selects one control appropriate for the current operating state of the VTR 40 from among a plurality of control commands in response to the instruction for specifying control of the operation mode recognized by the voice recognition circuit 13. It selects and outputs instructions. Details of this selection process and the operation mode of the VTR 40 will be described later. The control circuit 15 also outputs a message signal that instructs the optimal message as the response content according to the voice input content (and also according to the current state),
This message signal is sent to the animation character generation circuit 16 and the voice synthesis circuit 19.

Further, in FIG. 3, the transmitting section 11 of the handset 10 converts the voice uttered by a person (for example, the operator operating the VTR 40, etc., ie, the user) into an electrical voice signal. The voice uttered by the operator is used to directly control the various operation modes of the VTR 40, such as "play", "stop", "record", "
"Pause", "Slow", "Fast forward", "Rewind" and other commands are pronounced audibly, and the various information mentioned above includes, for example, the days of the week for recording reservations ("Sunday" to "Saturday", "
``Daily'', ``Weekly'', etc.), Channel (``1 Channel'')
- "12 channels" etc.), start/end time ("AM" etc.)
, "afternoon,""0:00" to "12:00," etc.) can be enumerated as vocal pronunciations. Furthermore, in this specific example, in addition to the above words, more human words such as "oi" are used.
It is now possible to input words such as , ``enough'', and ``no'' by voice.

[0040] The press talk switch 12 for specifying the voice input state disposed on the handset 10 is turned on/off by the operator to instruct a break between the words pronounced by the operator. It is. That is, the press talk switch 12 is provided to separate processing units of speech recognition in the speech recognition circuit 13 for an input speech signal consisting of a sentence composed of a plurality of consecutive discrete words. The output from the press talk switch 12 is sent to a switch state detection section 14 attached to the voice recognition circuit 13. This switch state detection section 14 forms a state instruction signal indicating the current on/off state in response to the output signal from the press talk switch 12. This state instruction signal may be, for example, state 0 when the press talk switch 12 is not activated (off), and state 0 when activated (
ON) is assumed to be in state 1. Therefore, if you want the voice recognition circuit 13 to perform voice recognition, turn on the press talk switch 12 and turn it off after voice input is completed, thereby processing the voice recognition based on the corresponding status instruction signal. Speech recognition processing is performed in the speech recognition circuit 13 in units of units. For this reason, when inputting continuous words by voice, the voice recognition circuit 13 does not have to analyze whether the input has ended due to the OFF state (state 0), which means the end of the voice word input. You will also be able to do it. In other words, in the speech recognition circuit 13, when recognizing speech, the clear start timing and end timing are known, and the range of speech recognition can be easily determined by software, so noise outside this range is ignored. This eliminates the need for unnecessary speech recognition processing. Furthermore, since the handset 10 is not controlled, noise is not generated when switching audio inputs (when audio is disconnected).

Here, the voice output signal from the voice synthesis circuit 19 is sent to the handset of the handset 10,
By inputting and outputting audio through the handset 10, which has a shape similar to that of a telephone, it is possible to reduce the feeling of resistance and discomfort when talking into an electronic device, and it is also possible to reduce the feeling of discomfort when talking into an electronic device. This will also help prevent malfunctions. Also, the speakers 2 of a television receiver, stereo set, etc.
By sending the response voice only to the receiver of the handset 10 without sending it to the receiver 10, the response voice can be shielded from the co-viewers, and recording reservation setting operations, VTR operations, etc. can be a nuisance to the co-viewers. It can be done without applying.

FIG. 4 shows a flowchart of the main operations in this specific example device. In FIG. 4, in an initial state such as after the device is powered on, when an operator (user) calls an animation character by performing a voice input or other input operation in step S1,
On the screen SC of the CRT display device 30, for example, FIG.
An animation character AC as shown in FIG. 2 is displayed, and the system enters a standby state for voice input by the operator. That is, on the screen SC of the CRT display device 30, as shown in FIG. The signal is displayed as a background image BG in a so-called superimposed manner. The message in the speech bubble SP (for example, the message "What do you want to do?" indicating the standby state) is simultaneously generated by the speech synthesis circuit 19 into synthesized speech with the same content, and is output to the speaker 20. This voice (for example, the voice "What do you want to do?") is pronounced. To give a more specific example of the calling process in step S1, when the name of the animation character AC (for example, Ivy) is called, or when a voice command such as "Hey" is input, or when the power of the VTR 40 is turned on, When turned on, the animated character AC is first displayed with the message "Yes, I'm Ivy," and then the state shifts to the state shown in FIG. 5, thereby creating a more natural dialogue atmosphere.

In the next step S2, the operator (
(user) inputs commands and various information for specifying and controlling the operation mode by voice. If this command is a command for operating the running system of the VTR 40, the process advances to step S3 and enters a phase in which the operation mode of the VTR 40 is directly controlled; if this command is a command for recording reservation, the process advances to step S4. Then, the VTR 40 enters the recording reservation operation phase. If the command is for confirming a reservation, the process advances to step S5 and enters a phase of confirming the contents of the reservation. In this reservation confirmation phase, after step S5, the process further advances to step S6 to change or cancel the reservation. In the above-mentioned recording reservation and reservation confirmation phases, the animation character A
A deeper level conversation with C takes place. When these steps S3, S4, and S5 (S6) are completed, the process returns to the voice input standby state of step S2.

In the example shown in FIG. 5, the animation character AC is displayed, for example, at the lower left of the screen SC of the CRT display device 30. That is, when the animation character AC is displayed in the center of the screen SC, the image (background image B) displayed on the screen
G), but if it is at the lower left, it is unlikely to interfere with the image on the screen SC. Moreover, if it is at the bottom of the screen SC, the feet of the animation character AC do not float in the air, giving a sense of stability, and if it is at the bottom left, there is less resistance from human sensibilities. The speech bubble SP for displaying a message on the screen SC should be placed at the bottom of the screen SC if the text string is written horizontally from left to right, and placed at the bottom of the screen SC if the text string is written vertically from top to bottom. For example, it is placed on the right side of the screen SC. Thereby, the image (background image BG) on the screen SC is less likely to be obstructed.

[0045] The motion of the animation character AC is as follows:
The animation operation control signal from the control circuit 15 is sent to the animation character generation circuit 16 for control. That is, the animation character generation circuit 16 receives the operation control signal from the control circuit 15.
is an animated character A as shown in FIG. 5 described above, which is a speaker for interacting with the operator.
A video signal of a moving image in which the mouth moves in response to the audio output of the message is outputted from C and supplied to the superimposer 17. This animation signal also includes a display signal for a speech bubble SP that displays the above message in text. The superimposer 17 converts the image of the animation character AC onto a reproduced image (or television broadcast image) from the VTR 40 into a video signal, so-called superimposing, and sends the superimposed signal to the CRT display device 30. Note that the screen S of the CRT display device 30
The animation character AC projected on C is
It is made to be anthropomorphic with facial expressions, making it more familiar. This allows the operator (user) to feel as if he or she is interacting with the animated character AC (or electronic device).

Next, the speech recognition circuit 13 in FIG.
The system recognizes the above-mentioned commands and various information by signal processing the supplied audio signal, and various configurations are possible, but the configuration shown in FIG. 6, for example, will be described. In this FIG. 6, the input terminal 21 is connected to the handset 1.
An output signal (input audio signal) from the transmitting unit 11 of 0 is supplied, and this input audio signal is sent to the arithmetic processor 23 via the analog interface 22 . The analog interface 22 controls the input audio level to a predetermined value in accordance with control data supplied from the system controller 24, converts it into a serial digital audio signal, and sends it to the arithmetic processor 23. The arithmetic processor 23 forms a voice pattern by, for example, frequency-analyzing the input digital voice signal, and also corrects temporal distortion of the voice pattern due to fluctuations in the speaking rate of human voices (time axis normalization). Then, the time axis normalized audio pattern is compared with a plurality of standard patterns stored in advance in the standard pattern registration memory 25 to perform a so-called pattern matching process. This pattern matching process is to calculate the distance between the detected audio pattern and each standard pattern to find the standard pattern with the shortest distance, and the process result is sent to the control circuit 15 comprising the CPU etc. sent to. Here, in the standard pattern registration memory 25, a plurality of audio patterns are registered in advance, such as command words such as "play" and "stop", and words of various information such as "Sunday" and "1 channel". The arithmetic processor 23 recognizes words and the like by determining which of the plurality of registered patterns (to which one) the voice pattern of the input voice signal is closest. In addition, the standard pattern registration memory 25
Pattern registration can be done by having the manufacturer memorize a plurality of standard voice patterns in advance, or by having the user input multiple words sequentially by an operator before starting to use the product, and by having the arithmetic processor 23, for example, register the patterns. The voice patterns may be formed by frequency analysis and stored.

The output signal from this speech recognition circuit 13 is
The information is sent to the control circuit 15, where it is subjected to natural language input processing and inference/dialogue processing using a predetermined software program. That is, as shown in FIG. 7, the natural language input processing production system 32 functions in response to the user's voice input 31 to form, for example, semantic frames 33 corresponding to the number of reserved programs, and these semantic frames 33 is sent to the inference/dialogue production system 34 to set and control the recording reservation scheduler 35. The natural language input processing production system 32 includes a sentence normalization production system PS1,
The inference/dialogue production system 34 can be subdivided into a segmentation production system PS2, a word extraction production system PS3, and a semantic understanding production system PS4. Processing production system PS
It can be subdivided into 7. One semantic frame 33 is provided with slots for a plurality of items, such as a day of the week information item, a channel information item, a start time information item, and a recording time or end time information item, for example, for recording reservation processing.
Various information input by voice during the reservation process is written into these corresponding item slots.

Next, FIG. 8 is a flowchart for explaining in more detail the processing at each step in the flowchart shown in FIG. . That is, in this FIG. 8, in step S11, the operation for calling the animation character AC in step S1 in FIG. 4 is performed. The call in step S11 is
As explained in step S1 of FIG. 4 above, this is done in response to the operator's voice input of a calling command (such as "Ivy" or "hey") or when the VTR 40 is turned on. When the input to call the animation character AC is made in this step S11, the process proceeds to the next step S12, where the animation character AC as shown in FIG. AC start).

Next, in step S13, it is determined whether the press talk switch 12 is on or off. If it is off (No), the process proceeds to step S14, and if it is on (Yes), the process proceeds to step S15. Step S1
When the command from the operator is input by voice in step 5, first, in step S16, it is determined whether or not this input command is a recording reservation command. If it is a recording reservation command (Yes), recording reservation processing is performed in step S17, and then the process returns to step S15. If No in step S16 (if the command input is not a recording reservation command), the process advances to step S18, where it is determined whether or not the command is a display command to display the contents of a recording reservation that has already been made.
In the case of s, the process returns to step S15 after displaying the contents of the previous recording reservation on the screen SC of the CRT display device 30 in step S19, and in the case of No, the process proceeds to step S20. In step S20, it is determined whether the command input by the operator is a command to change the recording reservation. If Yes in step S20, the recording reservation is changed in step S21, and then the process returns to step S15; if No, the process proceeds to step S22. In step S22, it is determined whether or not it is a command to cancel the recording reservation. If Yes, the recording reservation is canceled in step S23, and then the process returns to step S15. If No, the process advances to step S24. In step S24, it is determined whether or not this is an operation command for the animation character AC, and Y
In the case of es, the animation character AC is operated in step S25, and then the process returns to step S15. As a specific example of the operation of the animated character AC, there can be mentioned an operation such as making the animated character AC disappear from the screen, make it silent (mute the sound), etc., as will be described later. Moreover, in the case of No in step S24, the process advances to step S26. In step S26, it is determined whether or not the instruction is to operate the VTR 40. If YES, the VTR 40 is operated and then the process returns to step S15; if NO, the process ends.

[0050] If the answer in step S13 is No and the process proceeds to step S14, if there is no voice command input from the operator (user) within a certain period of time in step S14, for example, the animation character AC The video depicts a person who has too much time to spare. Specifically, for example, first the animated character AC yawns, then leans to the edge of the screen,
The animation character AC is made to make a motion of scratching its head, and if there is no voice command input continuously, the animation character AC is made to make a motion to waste time such as lying down (sleeping on its side). This motion processing will be referred to as free animation processing hereinafter.

FIG. 9 is a flowchart for explaining details of the reservation process in step S17 of FIG. 8 above. That is, in FIG. 9, in step S50, a reservation input request display indicating that information input for recording reservation is being waited for is displayed. For example, as shown in FIG. 10, the message "Please make a reservation" is displayed on the screen SC of the CRT display device 30 using an animation character AC and a text message in a speech bubble SP. At this time, the synthesized voice pronounces "Please make a reservation," and a moving image is displayed, such as the animation character AC's mouth moving. In the example of FIG. 10, for example, characters "reservation" indicating the current phase are displayed in the character display window PH at the upper right position of the screen SC.

In the state where the display (and audio output) as shown in FIG. 10 is made, the next step S5 in FIG.
1, various element information for recording reservation of the VTR 40, such as day of the week, channel, start and end times, etc., are input by voice. Here, the step S5
When inputting element information in 1, multiple pieces of element information can be input at once and in any order. For this reason,
In step S52, input processing is performed by the natural language input processing production system 32 as shown in FIG. The element information is classified and written into slots corresponding to each item of the meaning frame 33, such as a day of the week information item, a channel information item, a start time information item, a recording time information item, or an end time information item. After element information input processing is performed in step S52, it is determined in step S53 whether or not there is insufficient element information.

[0053] This lack of element information means that all the item information in the above-mentioned meaning frame is not available. In other words, for example, if four items are provided in the semantic frame at the time of recording reservation, a day of the week information item, a channel information item, a start time information item, and an end time (or recording time) information item, these four items If even one item of information is missing, it will not be possible to make a normal recording reservation. Therefore, in step S53, element information is insufficient (Yes).
), a question is asked about the missing element information in step S66, and the process returns to step S51. This missing information complementing process will be described later with reference to FIG. 13.

In step S53 of FIG. 9, there is no shortage (No.
), the process advances to step S54 and each input element information is confirmed. This step S5
When checking the element information in step 4, for example, use the CRT display device 3.
A display like that shown in FIG. 11 is made on the screen SC of 0. As for the display on the screen SC in FIG. 11, as in the case of FIG. ``Is this okay?'' is similarly pronounced and the animation character AC's mouth is displayed moving. Also screen S
"Reservation" indicating the current phase on the character display window PH of C
is displayed. Furthermore, in the confirmation display of FIG. 11, a recording reservation content display window PR is provided in the center of the screen SC, and within this display window PR, for example, the day of the week data of the recording reservation "Wednesday, November 7th" is displayed. , start and end time data "3:00 a.m. - 4:00 a.m.", and channel data "6 channels".

While the display shown in FIG. 11 is being performed, in the next step S55 in FIG.
4, a voice input is made to confirm the reservation. for example,
Commands such as "yes", "go back", "no", and "change" or the above-mentioned element information are input by voice. Of course, the natural language input processing shown in FIG. 7 described above is also performed at this time. In step S56, it is determined whether the voice input in step S55 is "yes" or not. If it is determined to be yes (input voice of "yes"), the process advances to step S67 and a check is made to see if there are any duplicate recording reservations. Will be checked. If the determination in step S56 is No (other than "yes"), the process advances to step S57. In step S57, it is determined whether the voice input in step S55 is "Go back", and if this determination is Yes, it is further determined in step S68 whether or not a question has been asked about the missing information in the element information input. . This step S
If 68 is Yes, the process returns to step S66, and if No, the process returns to step S50. On the other hand, if the determination in step S57 is No, the process advances to step S58. In step S58, it is determined whether the voice input in step S55 is element information. If the audio input is element information (Yes), step S52
If it is another voice input that is not element information (No.
), the process proceeds to step S59. In step S59, it is determined whether the voice input in step S55 is "change". If No, the process advances to step S60. In this step S60, a selection is made as to whether to change or cancel each element information of the recording reservation. For this reason, step S61
Enter either change/cancellation in step S62.
Proceed to. In step S62, the voice input is "changed" again.
It is determined whether or not, and if No, step S6
9 to cancel the recording reservation. Moreover, in step S62,
In the case of s, the process advances to step S63, where a question is asked about the content of the change, and after the element information is input again at step S65, the process returns to step S52.

FIG. 12 is a flowchart showing details of the element information input processing in step S51 and step S52 in FIG. 9 above. FIG. 12 shows the flow of operations focusing on the fact that the natural language "go back" has multiple meanings, and that different controls are required depending on the current state. That is, in FIG. 12, when element information is input, the input element information is semantically analyzed in step S71, and then the process proceeds to step S72. Step S7
In step 2, it is determined whether the input voice is "go back", and if YES, the process advances to step S73, where it is determined whether the immediately previous voice input was "go back". be exposed. If Yes in this step S73, the process advances to step S75, and after the reservation cancellation process is performed, the process ends. Further, if it is determined No in step S73, the process advances to step S74, and in step S74 it is further determined whether or not there was a previous question. If No in step S74, the process proceeds to step S75 to cancel the reservation, and if Yes, the process proceeds to step S76. In step S76, the previous question is asked, and the process returns to step S51 of FIG. 9, which is the element information input step. Further, if the determination in step S72 is No, the process advances to step S77. In step S77, it is determined whether or not there is an error, and if YES, the process advances to step S79, where a question regarding the error item is asked, and then step S5 is performed.
Return to element information input in step 1. No in step S77
If it is determined that this is the case, the process proceeds to step S78, and after writing to the meaning frame is performed, the process ends.

[0057] In addition to such words as ``go back'', when a word with multiple meanings is input by voice, such as ``cancel'' or ``good'', it is necessary to correctly understand the meaning depending on the situation. Natural language processing programs are designed to perform processing, that is, to handle the ambiguity of words. This can be achieved by incorporating the current state into the judgment target, as shown in FIG. 12 above, and thereby the operation can be performed with a natural feeling that is normally performed in human language life. Note that even in the phase of directly operating the running system of the VTR 40 in step S3 of FIG. has various meanings, and appropriate responses are taken depending on the situation, which will be discussed later.

Next, FIG. 13 is a flowchart showing details of the missing information processing in FIG. In order to simplify the illustration, in Figure 9, the presence or absence of various types of missing information and questions are shown together in one step, but in Figure 13, these are expanded according to the four items of the above meaning frame, and the recording reservation Steps are also added depending on the presence or absence of elemental information (hereinafter referred to as HLS data) obtained by learning habits. That is, in the missing information complementing process shown in FIG. 13, in the first step S33, it is determined whether or not there is an item of recording day of the week information, and if it is present, the process advances to step S36, and if it is absent, the process advances to the next step S34. . In step S34, a question is asked as to which day of the week the recording reservation should be made, using the video display and audio of the animated character AC.
In step S35, day of the week information is input. In step S36, it is determined whether there is an item of channel information for recording reservation. If it is present, the process advances to step S39, and if it is absent, the process advances to step S37. Step S37
Then, a question is asked as to which channel to select, and channel information is input in step S38.

[0059] In this specific example, the operator
It has a system (HLS) that learns reservation habits when making recording reservations at 40. In other words, in this HLS system, for example, when information about the day of the week and the channel (or only the day of the week) is input, in step S39, the information about the day of the week and the channel is added to the habitual recording reservation data (HLS If the data is HLS data, at least the start time and end time of the habitually reserved program are displayed (and audio output) in step S46, and this Confirm whether or not to make a habitual recording reservation. When such a display and confirmation is being made, "yes" or "no" is input by voice in step S47, and the process proceeds to the next step S48.
Determine whether or not to perform the habitual recording reservation described above, and select Y.
In the case of es (voice input of "yes"), step S49
If the answer is No, the process advances to step S41.

If it is determined in step S39 that the day of the week and channel information is not the customary HLS data, the process advances to step S40. In step S40, it is determined whether there is information on the start time of the recording reservation.
If there is, the process proceeds to step S43; if not, the process proceeds to step S41. In step S41, a question is asked as to what time the recording reservation should start, and in step S42, information on the start time is input. When this start time information is input, in step S43, it is determined whether there is information about the end time or recording time of the recording reservation. If there is, the process advances to step S49; if not, the process advances to step S44. In step S44, a question is asked about the end time of the recording reservation.
In step S45, time information of the end time or information of the recording time is input. In step S49, for example, a display as shown in FIG. 11 (and "Is this OK?") is displayed.
(audio output) is performed, and it is confirmed whether each piece of information for recording reservation inputted in each step above is correct.

Next, FIG. 14 is a flowchart for explaining details of the duplication check process in step S67 of FIG. In this FIG. 14, in step S81, the above-mentioned confirmation of reservation details is displayed (step S54 in FIG. 9 above).
(equivalent to steps S55 and S56 in FIG. 9) is performed, and in step S82, "yes" is input by voice input to start the duplication confirmation process (equivalent to steps S55 and S56 in FIG. 9). Therefore,
The steps after step S83 correspond to the duplication check process of step S67 in FIG. 9 above. In step S83 of FIG. 14, it is determined whether or not there is any reservation information that overlaps in time among the recording reservation information that has already been reserved. If No, the recording reservation is completed in step S92 and the process ends. , in the case of Yes, the process advances to step S84. In step S84, duplicate recording reservations are displayed and a selection is made to change or cancel the contents of the duplicate recording reservations. In step S85, "change"/"cancel" is selected by voice input.
Enter any of the following. In step S86, it is determined whether or not the voice input is "change". If No, the recording reservation is stopped in step S93 and the process ends; if Yes, a question is asked about the content of the change in step S87. do. Following this step S87, element information is input by voice in step S88. If element information is input in step S88, semantic analysis processing is performed in step S89. At this time, in step S90, it is determined whether there is an error in the semantic analysis of the voice input, and if there is an error, an error item is asked in step S94, and step S88
Return to If it is determined in step S90 that there is no such thing, the process advances to step S91, where slot writing is performed to the corresponding information item of the meaning frame, and then the process returns to step S81.

Here, the error processing in steps S90 and S94, and the error processing in steps S77 and S79 in FIG. , a syntax error occurs when an unmatch occurs in the semantic analysis routine (there is no rule), and an input error occurs when there is an error in the specification of element information (or there is no error in the specification, but it is incorrect due to the VTR specifications). Can be mentioned. In the case of the above-mentioned grammar error, for example, a message such as "I didn't understand. Please say it again." is displayed as a video and outputted as audio, and the system waits for input. In the case of the above input error, for example, "I don't understand. What day of the week is your reservation?" or "The start and end times are the same. Please re-enter the time." Displays a video and outputs a message pointing out the error item (error item) and requesting re-input, and waits for input.

Next, FIGS. 15 and 16 are flowcharts for explaining details of the display processing routine in step S19 of FIG. 8. That is, when the display process in step S19 of FIG. 8 is started, first the display process shown in FIG.
In step S100, a voice input such as a display command, change command, cancellation command, etc. is performed. In step S101, it is determined whether there is recording reservation information or not, and if there is no recording reservation information,
In step S102, an error message indicating that there is no reservation information to be displayed is displayed as a video and output as audio, and the process ends. If it is determined in step S101 that the recording reservation information exists, the content of the recording reservation is displayed in step S103, and then it is determined in step S104 whether there is undisplayed reservation information. If it is determined in step S104 that there is no such information, the process advances to step S113. If it is determined in step S104 that the screen is to be switched, a question is asked in step S105 as to whether the screen of the CRT display device 30 should be switched. In step S106, a voice ``yes'' or ``no'' is sent.
, "Go back," an instruction to change, or an instruction to cancel is input. After the input in step S106 is made, it is determined in step S108 whether or not the input voice is "yes". If it is determined Yes in this step S108, the process advances to step S115, and it is further determined whether there is information for the next screen. This step S1
If it is determined Yes in step S15, the process advances to step S116, displays the next screen, and returns to step S105. If it is determined No in step S115, the process advances to step S117, displays an error message, and returns to step S105. return. Further, if the determination in step S108 is No, the process advances to step S109. The step S1
At step S09, it is determined whether or not the voice input is "go back", and if YES, the process advances to step S118. In step S118, it is determined whether or not there is a previous screen. If Yes, the previous screen is displayed in Step S119 and the process returns to Step S105; if No, the process ends. If the answer in step S109 is No, the process advances to step S110. In step S110, it is determined whether or not the voice input is a change command. If Yes, the process proceeds to step S120, and if No, the process proceeds to step S111. In step S111, it is determined whether the voice input is a cancellation command. If Yes, the process proceeds to step S121, and if No, the process proceeds to step S112. In step S112,
It is determined whether or not these change processing and cancellation processing started from the display command. If No, the process returns to step S110, and if Yes, the process proceeds to step S113. In step S113, a question is asked as to whether the change or cancellation is to be made, and in step S114, a voice response of "yes" or "
input "No", "Go back", change command, or cancellation command.

Furthermore, when the process of step S114 in FIG. 15 is completed, the process advances to step S125 shown in FIG. In step S125, it is determined whether the voice input in step S114 is a change command, and
In the case of s, the post-processing after performing the change process in step S131 is ended, and in the case of No, the process proceeds to step S126. In step S126, it is determined whether the voice input in step S114 is a cancellation command, and if YES, the cancellation process is performed in step S132, and then the process is terminated.
If No, the process advances to step S127. The relevant step S
127, the voice input in step S114 is "yes"
A determination is made as to whether or not this is the case, and if the determination is No, the process ends; if the determination is Yes, step S128
Proceed to. In step S128, a question is asked to select change/cancellation, and in step S129, a change command, a cancellation command, and a "return" command are input by voice. Thereafter, the process advances to step S130, where it is determined whether the voice input is "go back". If the determination is Yes, the process ends; if the determination is No, the process returns to step S125.

FIG. 17 is a flowchart for explaining details of the change processing routine in step S21 of the flowchart of FIG. 8. That is, in FIG. 17, in step S140, a recording reservation selection process is performed, and specifically, for example, a list of recording reservation contents is displayed, and a message asking which reservation to change is displayed in the video display and audio. Output. In the next step S141, if there are, for example, four content display columns in the displayed list, the number of columns among these columns in which the recording reservation information is to be changed is determined, for example, from "No. 1" to "4". Enter by voice, such as "No.". In the next step S142, it is determined whether or not there is information about actual recording reservation in the column of the number specified by the voice input. If No in this step S142, an error message is output in step S150 and the process returns to step S140, and if Yes, the process proceeds to step S143.

[0066] In step S143, a question is asked about the content of changes to the specified recording reservation content, and in step S144, element information of the content of changes is input by voice. In step S145, the recording reservation content changed based on the element information is confirmed, and in step S146, "yes" or "no" or the element information is input by voice. In step S147, the step S
A determination is made as to whether or not the voice input No. 146 is element information. If Yes, the process returns to step S145; if No, the process proceeds to step S148. In this step S148, it is determined whether the voice input is "yes" or not, and Y
In the case of es (voice input of "yes"), step S15
After performing the above-mentioned duplication check process in step 1, the process ends. Further, if the determination in step S148 is No,
After the question about the change details is asked again in step S149,
Return to step S144.

FIG. 18 is a flowchart for explaining details of the cancellation processing routine in step S23 of the flowchart shown in FIG. That is, in FIG. 18, in step S160, the recording reservation information to be canceled is selected, and in step S161, for example, 4
The user inputs by voice which recording reservation information among the available recording reservation information is to be canceled. In step S162, this number 1~
It is determined whether recording reservation information No. 4 exists. If No in this step S162, an error message is displayed in step S168 and the process returns to step S160.
In this case, the process advances to step S163. The processing from step S160 to step S162 in FIG. 18 is substantially the same as the processing from step S140 to step S142 in FIG. 17 described above.

[0068] Next, in step S163, cancellation confirmation processing is performed, and in step S164, a voice ``yes'' is confirmed.
, "No" is input. In step S165,
A determination is made as to whether or not the voice input in step S164 is "yes", and this determination is Yes (the voice input is "yes").
If the answer is "Yes"), the process advances to step S166, and the post-processing that has been determined to be canceled is completed. If the determination in step S165 is No, the process advances to step S167, cancels the cancellation process, and ends the process.

In the phase of recording reservation or reservation confirmation (and change/cancellation) as explained above, the process shown in FIG.
Steps S54 to S59 of , and step S of FIG.
105 to S111 and steps S113 to S113 in FIG.
As is clear from step S127 in FIG. 15 and steps S145 to S148 in FIG. 17, input other than the requested answer is also accepted and understood. That is, in step S54 of FIG. 9, a display (and voice output) as shown in FIG. By inputting the element information by voice, it is determined Yes in step S58 and the element information input process S is performed.
52. In addition, by inputting "change" by voice, it is possible to proceed to the change process. Also, Figure 1
In Step S105 of Step 5, a question message asking whether to switch the screen is displayed as a video and is outputted as audio. Normally, the answer is "yes" or "no," but a change command or cancellation command can be directly input. Then, the process can proceed to change processing step S120 or cancellation processing step S121.

[0070] This is required in consideration of the fact that in normal conversations between humans, the meaning may be inferred from the context and a dramatic response that does not formally follow the other person's previous statement may be made. It also accepts input other than the given answer.
This allows optimal processing to be performed depending on the input content. This makes it possible to directly enter the next input step by omitting, for example, a "yes" or "no" answer, thereby simplifying the operating procedure. Furthermore, there are multiple procedures for moving to a predetermined process. For example, if you want to change the recording reservation details, answer "No" to the question "Are you sure you want to do this?" during confirmation, and then "What do you want to do?"
The orthodox procedure for responding to each question is to order "change" in response to the question "Change", but when confirming the above,
You can enter commands such as , enter changes directly, or press ``
It is possible to return to the previous information input state by commanding "Go back," and it is possible to flexibly respond to various user (operator) inputs. What's more, it is also possible to change the element information of the change item by directly inputting it, such as "6 channels," which not only simplifies the operating procedure, but also allows dialogue in natural language without mentioning obvious things. An environment or atmosphere can be realized.

FIG. 19 is a flowchart for explaining details of the operation processing routine for the VTR 40 in step S26 of the flowchart shown in FIG. That is, in this FIG. 19, in step S171, the VTR 40
The current state (for example, which mode of operation is set) is checked. In step S172, a matrix (command-
Search from the operation comparison table). A portion of this matrix is shown in Table 1 below.

[Table 1] In the next step S173, it is determined whether or not there is a corresponding operation in this matrix. If there is no corresponding operation, an error message is output in step S176 and the process ends. Moreover, if there is no command in step S173, the process advances to step S174, and in step S174, the operation corresponding to this command is executed. After that, step S17
In step 5, a message indicating that the process is to be executed or has been executed is output and the process is terminated.

[0072] Here, Table 1 above shows an example of which operation mode the current operation mode will shift to when the above-mentioned voice input command is received from the current operation mode. In other words, in Table 1, for example, when "power" is entered as a voice command, if the current operation mode is power off, the operation mode is controlled to power on, and the tape is then turned on in the same manner. In the case of none, stop, playback, etc., the power is turned off. Furthermore, when a voice command such as "playback" is input, if the current operating mode is power off, stop, etc., the operating mode is controlled to playback. Furthermore, for example, when inputting the word "quickly" by voice, if the current operation mode is stop, fast forward, or cue, it will be controlled to fast forward, and if it is playback or pause, it will be controlled to cue. ,
If it is a rewind, it is controlled to review, and if it is a review, it is controlled to rewind. Furthermore, for example, when inputting a voice command such as "Go back", if the current operation mode is stop, fast forward, or pause, the current operation mode will be controlled to review, and if it is playback or cue, it will be controlled to reverse playback, and if the current operation mode is playback or cue, it will be controlled to reverse playback. If it was a rewind, it would be controlled to fast forward, and if it was a review, it would be controlled to play. In this way, even in response to an ambiguous voice input such as "quickly" or "go back", a control action with the correct meaning is selected according to the operating mode at that time.

Next, FIG. 20 is a flowchart for explaining the details of the idle animation processing routine in step S14 of the flowchart of FIG. 8 above. In FIG. 20, in step S181, it is determined whether or not the level is the top level. This top level is VTR4
This refers to the level of conversation at which the basic operations of 0 are performed (directly) and the reservation/confirmation phase described above is entered. Yes in this step S181 (at the top level)
If it is determined that this is the case, in step S186 the video of the leisure animation for the top level is selected, and then in step S18
Proceed to step 3. Further, if the determination in step S181 is No, then in step S182, a video of a leisure animation for each phase-by-phase processing mode is selected. These leisure animation images for top level or processing mode can be set in various ways in advance, such as when the animated character AC yawns or leans toward the edge of the screen, for example, as mentioned above. , scratching its head, or even make the animation character AC do a relaxing action such as lying down (sleeping on its side). In step S183, a process is performed to select one of these free time animation video patterns by, for example, random number generation, and then the process proceeds to step S184. In step S184, it is determined whether a predetermined waiting time has elapsed when there is no voice input, and if No, this determination is repeated, and if Yes, the process advances to step S185. In step S185, the leisure animation video of the selected pattern is displayed as a moving image on the screen SC of the CRT display device 30. That is, the animation character AC is caused to perform an action indicating that it has too much free time.

[0074] Furthermore, in order to make the responses of the animation character AC more human-like and give a sense of familiarity, some kind of response (play response) is made even to voice inputs unrelated to specifying the operating state of the device. It is preferable to do so. For example, in response to a voice input of ``Hey,'' the response is ``Yes, what is it?'', or in response to a voice input of ``Okay, yoshi'', the response is ``You're embarrassed.'' For example, it may display animations such as drawing.

By the way, the control circuit 15, animation character generation circuit 16, and voice synthesis circuit 19 shown in FIG.
The interface between the two is performed using a message packet MP having a predetermined structure. A specific example of this message packet MP may include three elements: a message type indicating the type of message, a message number associated with each message content, and a status. The above message types include, for example, messages without input (see Figures 5 and 10 above), messages with input (see Figure 11 above), display of scheduler packets, and lists (list of reservation contents). Instructs the display of duplicate lists, display of duplicate lists, etc. For example, when the message number is 8 bits, it can specify one of 256 types of messages. That is, the control circuit 15 selects an appropriate message type and type (content) according to the recognition data of the input voice, the current state (status), etc., and inserts the message type and message number into the message packet MP. Then, it is sent to the animation character generation circuit 16 and the voice synthesis circuit 19. Animation character generation circuit 1
6 and the voice synthesis circuit 19 display the message according to the message number in the sent message packet MP as a video (display of the mouth movements of the animation character and the text of the message) or synthesize voice and output the message. That's why.

[0076] When displaying and audio outputting a message via such a message number, it is necessary to prepare several types of messages in advance, and among these multiple messages, One message is selected from the list to be displayed and output as audio, but instead, the control circuit 15 is configured to compose the message itself, and the created message is sent, for example, one character at a time, to create a video. Display and voice synthesis may also be performed.

Next, the above natural language input processing will be further explained. For example, when making a recording reservation, inferences regarding missing items such as time information are made using common sense. For example, if only minutes are specified when inputting the start time,
The time is estimated in relation to the current time, or if hours or minutes are input when inputting the end time, the recording time is set as the recording time from the start time. In addition, when multiple element information is input, considering the relationship between "from" and "until,"
It is estimated that the time is from 0 to 9:00, or the current time is 8:01.
If it is 0 minutes, the time is estimated to be from 8:30 to 9:30 in response to a voice input of "30 minutes to 1 hour."

Note that the present invention is not limited to the above-mentioned specific examples; for example, in addition to a telephone receiver, a so-called hand microphone may be used as the voice input means, or a small microphone may be used in a remote control device (so-called remote control). You may also provide a. Switch (press talk switch)
may be provided on these hand microphones or remote controls. Further, the electronic device to be controlled is not limited to a VTR, but can be applied to a control device for various electronic devices such as a disk recording and/or reproducing device, a digital or analog audio tape recorder, etc.

[0079]

[Effects of the Invention] As described above, the control device for electronic equipment of the present invention recognizes commands from the user of the electronic equipment, and uses the recognition results to infer or understand the commands from the user. By controlling the electronic device based on this result and generating a message to the user according to the inference result, the intent of the command from the user can be accurately analyzed, making it easy for users who are unfamiliar with operating electronic devices. Instead, it is now possible to accurately control electronic devices.

In addition, in the control device of the present invention, an agent that operates the device on behalf of the user is introduced into the human interface in order to make electronic devices that are complicated to operate easy to use even for users who are weak with machines. Furthermore, this agent is a metaphor for a person skilled in operating equipment, and in addition to having knowledge of equipment operation, it also has the ability to understand the user's intentions and carry out their intentions. This makes it possible to facilitate communication between users and devices. Therefore, even users who are not accustomed to operating electronic devices can now use the devices easily.

Furthermore, in the electronic device control device of the present invention, the basic flow of the interface, that is, the flow of conveying the intention in words and executing the desired operation on the electronic device by the user, can be completely controlled by simple instruction. It is easy to use because it can be used in everyday language with fewer restrictions on the words used, and even for people who cannot operate the device themselves or who are using this interface for the first time. It is very convenient. In addition, when the control device expresses human emotions, the user has a favorable impression, feels a sense of familiarity with the device, and is less likely to have a psychological reaction to communicating with a machine. It has a mitigating effect.

[Brief explanation of drawings]

FIG. 1 is a block diagram showing a schematic configuration of a control device according to an embodiment.

FIG. 2 is a diagram schematically showing an example device.

FIG. 3 is a block diagram showing a schematic configuration of a control device of this specific example.

FIG. 4 is a flowchart of main operations in the device of this specific example.

FIG. 5 is a diagram showing a CRT screen on which animation characters and speech balloons are displayed in an initial state.

FIG. 6 is a block diagram showing an example of the configuration of a speech recognition circuit.

FIG. 7 is a block diagram for explaining functions in a control circuit.

FIG. 8 is a flowchart showing details of processing at each step in the flowchart of FIG. 4;

FIG. 9 is a flowchart showing details of reservation processing in the flowchart of FIG. 8;

FIG. 10 is a diagram showing a CRT screen on which a reservation input request is displayed.

[Figure 11] CRT with display for confirming reservation information
It is a figure showing a screen.

12 is a flowchart showing details of element information input and element information input processing in the flowchart of FIG. 9; FIG.

FIG. 13 is a flowchart showing details of missing information processing in the flowchart of FIG. 9;

FIG. 14 is a flowchart showing details of the duplication check process in the flowchart of FIG. 9;

FIG. 15 is a flowchart showing details of display processing in the flowchart of FIG. 8;

FIG. 16 is a flowchart that is a continuation of the flowchart in FIG. 15;

FIG. 17 is a flowchart showing details of change processing in the flowchart of FIG. 8;

FIG. 18 is a flowchart showing details of cancellation processing in the flowchart of FIG. 8;

19 is a flowchart showing details of VTR operation processing in the flowchart of FIG. 8; FIG.

20 is a flowchart showing details of the leisure animation process in the flowchart of FIG. 8; FIG.

[Explanation of symbols]

50...user 60...Control device 61...Mike 62...Speech recognition section 63・・・Dialogue understanding department 64...Speaker 65...Speech synthesis section 66...Electronic equipment control section 67...Message generation section 68...Character display area 69...Monitor 71...VTR 73...Television receiver

Claims (1)

[Claims] Claim 1: A voice recognition unit that recognizes speech from a user using an electronic device based on a preset vocabulary information table, and at least the above-mentioned in association with control information for controlling the electronic device. a dialogue understanding unit that understands or infers the recognition result from the speech recognition unit and outputs the understood control information or inference result information; An electronic device comprising: a message generation section that generates at least message information for requesting confirmation; and a control section that receives the control information from the dialogue understanding section and controls the electronic device. Control device.