JPH07302351A - Picture and voice response equipment and method therefor - Google Patents

Abstract

PURPOSE:To provide a picture and voice response equipment capable of being operated as a substitution for a man and making an office work confortable by moving the mouth part of a human picture displayed on a display means in accordance with a voice output from a voice response means. CONSTITUTION:A user 11 sits on a chair oppositely to the display screen 13 of a video display part to be a virtual secretary. Namely, the existing direction of the user 11 is detected by a user detecting sensor, the screen 13 on a turn table 14 is turned to the direction of the user 11 based upon the detected result. The screen 13 is electrically connected to an equipment body 16 through a cable 15 and a communication line 17 is drawn out from the main body 16. A secretary picture is displayed on the screen 13 and its mouth part is moved in accordance with a voice output from a voice output means, so that the user 11 executes business processing such as data retrieval by a short instruction only by a message in an atmosphere as if the user 11 talks with a human secretary.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention is used as an office automation (OA) device, for example, to understand a word spoken by a user and actively provide information to the user. The present invention relates to an image / voice response method.

[0002]

2. Description of the Related Art Conventionally, OA equipment such as information equipment or office equipment has been provided to assist human work in an office.

[0003] For example, in a computer which is said to be closest to human brain labor, recent personal computers and "WINDOW
Providing more reference / work screens to the user or displaying more various so-called "multimedia" information, as represented by "S" (registered trademark of Microsoft Corporation). It is possible, and the pursuit of technology is progressing with a focus on better library and library direction.

[0004]

However, in the conventional OA equipment, "advanced OA equipment that acts as a human substitute"
It is not intended to aim at, but is made as a "machine that improves the human work environment", so there are still many human machine operations in the office, and it is still satisfactory from the viewpoint of comfortable office work. It wasn't something.

In view of the above-mentioned conventional problems, it is an object of the present invention to provide an image / voice response device and an image / voice response method which function as a substitute for humans and enable comfortable office work.

[0006]

In order to achieve the above object, an image / speech response device of the present invention includes a speech recognition means for recognizing language information included in a user's speech, and a recognition result of the speech recognition means. An image / voice response device comprising voice response means for voice-outputting the response content corresponding to, and display means for displaying image / document information, wherein the display means is a person image by computer graphics or a natural image. Is displayed and the mouth portion of the person image displayed on the display means is moved according to the voice output from the voice response means.

In the above-mentioned image / speech response device, a person image storage means for storing a plurality of types of person images is provided, and the person image selected by the user from these person images is displayed on the display means. You may do it.

In the above-mentioned image / speech response device, there is a user detecting means for detecting the physical presence of the user in the vicinity of the device, and when the user is detected by the user detecting means, the display is made. The display of the person image on the means may be turned on, and the display of the person image may be turned off when the user is not detected.

In the above-mentioned image / speech response device, the display of the person image on the display means is turned on by the voice uttered by the user, and the display is turned off after a lapse of a predetermined time after the voice is interrupted. Good.

In the above-mentioned image / speech response device, the user detection means is configured to detect the direction in which the user is present, and the display surface of the display means is set in the direction detected by the user detection means. You may turn it.

The above-mentioned image / sound response device has a sound level detection means for detecting the sound level of the user,
The voice response unit may output the response contents by voice at a voice level according to the detection result of the voice level detection unit.

In the above-mentioned image / speech response device, the voiceprint registration means in which the voiceprint pattern of the user is registered, the voiceprint analysis means for analyzing the voiceprint of the user, and the analysis result of the voiceprint analysis means is the voiceprint registration means. Determination means for determining whether or not it matches the voiceprint pattern in the inside, and permitting voice output by the voice response means when the determination means determines that they match, and the voice response means when it is determined that they do not match. It is also possible to provide a response control means for prohibiting the audio output by.

In the above-mentioned image / speech response device, when the voiceprint pattern is registered in the voiceprint registration means, a registration voice output means for outputting a registration voice and a voice uttered by the user following the registration voice. Registration control means for registering the voice print pattern of the above in the voice print registration means.

In order to achieve the above object, the image of the present invention
In the voice response method, a person image is displayed on the display means by computer graphics or a natural image, the language information included in the voice of the user is recognized, and the response content corresponding to the recognition result is output at the same time as the voice output. The mouth portion of the person image displayed on the display means is moved according to the sound output.

[0015]

According to the image / speech response device of the present invention having the above structure, the speech recognition means recognizes the language information included in the speech of the user, and the speech response means responds to the recognition result of the speech recognition means. The contents are output as voice. Then, the person image is displayed on the display means, and the mouth portion of the person image displayed on the display means is moved according to the voice output from the voice response means. As a result, the user can search for a material or the like with a slight verbal instruction in an atmosphere as if he / she were interacting with a human secretary.

By displaying the person image selected by the user from the person image storage means on the display means, the person image can be displayed according to the user's preference.

Further, the display of the person image on the display means is turned on when the user is detected by the user detecting means, and the display of the person image is turned off when the user is not detected. The device can be operated only when there are users nearby.

Further, by turning on the display of the person image on the display means by the voice uttered by the user and turning off the display after a lapse of a predetermined time after the voice is interrupted, the operation can be stopped when it is unnecessary. it can.

Further, by directing the display surface of the display means in the direction detected by the user detection means, the user is not restricted from moving within the office, for example.

Further, the voice response unit outputs the response contents by voice at the voice level corresponding to the detection result of the voice level detection unit, so that the user can interact with the apparatus at an appropriate voice level.

The voiceprint analysis means analyzes the voiceprint of the user, the determination means determines whether or not the analysis result matches the voiceprint pattern in the voiceprint registration means, and the response control means determines that they match. The confidentiality of the information in the device is maintained by permitting the voice output by the voice response unit at times and prohibiting the voice output by the voice response unit when it is determined that they do not match.

The registration voice output means outputs a registration voice when the voiceprint pattern is registered in the voiceprint registration means, and the registration control means uses the voiceprint pattern of the voice uttered by the user in accordance with the registration voice. Register to voiceprint registration means. As a result, the voiceprint pattern of the user can be registered easily and accurately.

According to the image / speech response method of the present invention, the person image is displayed on the display unit, the language information included in the voice of the user is recognized, and the response content corresponding to the recognition result is output as voice. At the same time, the mouth portion of the person image displayed on the display means is moved according to the sound output. This allows the user to feel as if they were interacting with a human secretary, for example, with a few verbal instructions.
You can search materials.

[0024]

Embodiments of the present invention will be described below with reference to the drawings.

FIG. 1 is a block diagram showing a schematic configuration of an embodiment of an image / voice response device according to the present invention.

This image / speech response device is composed of a computer device having a highly automated man-machine interface, and is used as a secretary robot (hereinafter,
It is used as a virtual secretary) and interprets a user's verbal command and automatically performs paperwork.

Reference numeral 1 in the figure is a CP for controlling the operation of the entire apparatus.
The CPU unit 1 is a U unit, and a file unit 3, a communication unit 4, an operation panel 5, an audio input unit 6, an audio output unit 7, and a video display unit 8 are interconnected to the CPU unit 1 via a bus line 2. There is.

The file unit 3 is composed of a RAM or the like, and stores various data such as text data, image data (still images and moving images), voice data, voiceprint patterns of users, and telephone directory data. The image data to be stored includes person images of a plurality of types of secretaries (hereinafter referred to as secretary images). As a secretary image, a natural image (photo / movie image)
People, animated images created by computer graphics, and other fictional character images are used. In addition, clothes images for this secretary image can also be used for women and men in a variety of business and formal styles. Has been saved. Also, as voice data,
Several types of voice patterns are stored for men, and the information of the person who is familiar to the user is additionally registered at any time.

The communication section 4 has a modem for modulating transmission image data output to the communication line 4A and demodulating analog data from the communication line 4A, and a network control unit (NCU) for switching lines. There is.

The operation panel 5 is composed of various operation switches and the like, and the voice input section 6 for collecting the verbal instruction of the user uses a directional sound collecting microphone in order to better pick up the user's voice. In addition, a filter mechanism for cutting out ambient noise is also provided. Furthermore, the voice input unit 6 is provided with a voice recognition device such as a specific speaker system, and the voice recognition device recognizes language information included in the voice of the user collected by a microphone or the like, and a text sentence that can be processed by a computer. Convert to code information such as.

The voice output unit 7 is composed of a voice response device, a speaker, a voice level detecting means, etc., and converts information (response contents) such as text sentences stored in the file unit 3 into voice by the voice response device, and the speaker Output by. Here, the voice level detection means detects the voice level of the user, and the voice output section 7 outputs the voice from the speaker at the voice level according to the detection result of the voice level detection means. There is.

The video display unit 8 is composed of a liquid crystal display device or the like, and displays a secretary image by computer graphics or a natural image and a document.

The CPU unit 1 has a ROM for storing control programs and the like, and a RAM for storing the calculation results and the like.
Also, an input circuit, an output circuit, etc. to which the user detection sensor 1A and the like are connected are provided, and various kinds of control as follows are performed according to a control program.

The user detection sensor 1A is for detecting the physical presence of a user in the vicinity of the apparatus, and is installed at a predetermined location in the office. The CPU unit 1 turns on the display of the secretary image on the video display unit 8 when the presence of the user is detected by the user detection sensor 1A, and turns off the display of the secretary image when the user is not detected. Further, when the user is making a sound, the display of the secretary image on the video display unit 8 is turned on, and the display is turned off after a lapse of a predetermined time after the sound is interrupted. Further, the mouth portion of the secretary image is moved according to the voice output from the voice output means 7.

Further, the CPU section 1 analyzes the voiceprint of the user input from the voice input section 6 and judges whether or not the analysis result matches the voiceprint pattern in the file 3,
When it is determined that they match, the voice output by the voice output unit 7 is permitted, and when it is determined that they do not match, the voice output is prohibited.

Although not shown, computer peripherals such as an image reader, a printer and an optical character reader (OCR) are connected to the apparatus.

FIG. 2 is a conceptual diagram showing a state during operation of the virtual secretary in this embodiment.

In the figure, 11 is a user, and the user 11 sits on a chair 12 and faces the display screen 13 of the video display section 8 of the virtual secretary. That is, the user detection sensor 1
The direction in which the user is present is detected by A, whereby the display screen 13 is rotated on the turntable 14 to face the direction of the user 11.

The display screen 13 is electrically connected to the device body 16 through a cable 15, and a communication line 17 is drawn from the device body 16.

Then, the secretary image is displayed on the display screen 13, and the part of the mouth thereof moves in accordance with the voice output from the voice output means 7 in an atmosphere as if the user is interacting with the secretary. The user can perform paperwork such as searching for materials with only a few instructions given orally.

As described above, the virtual secretary of the present embodiment performs the interface with the user by "the virtual secretary image created by the computer on the screen" and "the voice conversation". During the conversation with the virtual secretary and the user's voice, the virtual secretary controls the display screen 13, the sound collecting microphone, and the speaker so that they are always oriented in the maximum voice direction, and the virtual secretary is in the office of the user. Follow the position movement in.

Next, an example of office processing using the virtual secretary of this embodiment will be described.

First, as an initial setting of the virtual secretary, (1)
The secretary image is set, (2) the voice quality of the virtual secretary is set, (3) the voice level is set, (4) the name of the virtual secretary is set, and (5) the voiceprint of the user is registered.

Regarding the setting of the secretary image of (1), a plurality of types of person images registered in the file section 3 are displayed on the display screen 13 and the user is prompted to input a voice or select the operation panel 5. For example, when the user is a male, a female video sample, and when the user is a female, a male video sample is automatically displayed on the display screen 13 so that the user's favorite video can be selected. To do. Alternatively, a plurality of secretary images may be selected and used by switching them at each time of day, daily or monthly.

Similarly, the clothes worn by the secretary are selected from the file section 3 by the user.

Then, for example, the final secretary image is set by synthesizing the secretary image and the clothes image on the display screen 13 and changing the clothes and allowing the user to select a desired combination while displaying.

Regarding the setting of the quality of the voice generated by the virtual secretary in (2), the virtual secretary outputs the voice patterns of several kinds registered in the file section 3 through the voice output section 7 to the user's liking. Let them choose one. Specifically, the virtual secretary side utters a plurality of female voice samples and male voice samples as a menu according to the set distinction between female and male in the secretary image, and allows the user to select one. .

The voice level is set in (3) in consideration of the fact that the optimum voice volume varies depending on the size of the office, the hearing ability of each user, and the like. The user continuously utters a specific word or sentence, and the user verbally gives a “larger” or “smaller” adjustment instruction. Also,
The adjustment instruction may be issued from the operation panel 5.

The name of the virtual secretary (4) is set by designating a specific name from the user side.
Specifically, it is performed by inputting a word by voice from the user side or by inputting from the operation panel 5. Also, a plurality of names may be registered in the file unit 3 in advance, and the user may select from among them. In this embodiment, the virtual secretary is given the name “Mariline”.

The registration of the voiceprint of the user of (5) is for the purpose of protecting the confidentiality of the information stored in the device by not responding to a human voice other than the registered voiceprint, and for the specific speaker method. This is done for the purpose of using this registered voice sample when using voice recognition.

Specifically, the voiceprint pattern of the user is registered in the file unit 3 in such a manner that the user speaks first (voice for registration), and the user subsequently utters the same word.

When the above-mentioned initial setting is completed and the user comes to the office, the virtual secretary senses the presence of the person in the office by the user detection sensor 1A, and becomes the operation start state.

In response to the greeting of the user "Good morning, Marilyn", when the voice quality analysis confirms that it is the same as that of the user, the greeting "Good morning. XXX" is returned.

The information of the FAX and the electronic mail which is automatically received last night is stored in the file section 3, and the OCR
Has been converted to text data.

From the virtual secretary side to the user, "XXX
FAX has arrived from Mr. X. , "A mail has arrived from Mr. XXXX."
In response to the verbal instruction of “read me aloud,” the virtual secretary will convert these text sentences into voice by the voice response unit of the voice output unit 7 and start reading aloud.

When the "schedule" and the virtual secretary are verbally instructed, the voiceprint of the user is recognized, and by checking the agreement with the voiceprint data registered in the file section 3, it is confirmed that the user is the master. , Today's schedule table data is extracted from the file unit 3, converted into voice, and read aloud. At this time, if the video information of the person to be visited is registered in the file section 3, it is simultaneously displayed on the display screen 13.

When "Type" is designated, the virtual secretary is in the position of a dictation word processor. Thereafter, the words spoken by the user are heard by the virtual secretary, converted into text data by the voice recognition device, and stored in the file unit 3.

Further, if "telephone, XXXX" is instructed, the virtual secretary will select X from the telephone directory data in the file section 3.
The telephone number of Mr. XXX is searched for, and the communication section 4 has a modem function to automatically dial. Then, when the call is connected and the other party answers,
Give a voice response with "I was connected with Mr. XXXX".

After that, the voice signal of the user is poured into the telephone signal by the sound collecting microphone of the voice input unit 6, and the voice of the other party of the telephone is transmitted to the user through the speaker of the voice output unit 7 to make a freehand. Telephone conversation is conducted.

If the phone number of Mr. XXXX does not exist in the phone book data, "Sorry,
What is your phone number? , "And the user is asked to input the telephone number verbally, and then the dialing operation described above is started. This telephone number is thereafter additionally stored in the telephone directory data in the file section 3 and used as one of the subsequent telephone directory data for automatic dialing.

When a verbal command "FAX, XXXX" is issued, the FAX number of Mr. XXXX registered in the telephone directory data in the file section 3 is confirmed, and then "What's the content?" Give a reply. In response to this, when the instruction for "dictation" is given, the virtual secretary then enters a listening posture, listens to the user's speech, converts this into text data, further develops the font into a character image, and then the FA
The X image is transmitted to the other party (Mr. XXXX) through the communication unit 4.

Further, when the verbal command "put out file" is given, the virtual secretary asks "What is the title and keyword?", So an appropriate word is answered. The virtual secretary takes out some of the data that has already been created and stored from the file section 3 and inquires as to "...?". Therefore, when the operator selects the desired one, the same fax Send to the other party by action.

When the "type" is designated, the virtual secretary thereafter enters a listening posture, listens to the user's speech, converts it into text data, and files it in the file unit 3.

In the present embodiment, when displaying information such as a document for the user on the display screen 13, it is necessary to display the secretary image and information on the display screen 13. Has a function of (a) switching and displaying a secretary image and an information image, and (b) having a function of displaying two types of methods on the video display unit 8 and displaying the secretary image and the information video separately, (c) ) A method such as screen division display by one method can be considered.

When adopting the display method of (c), the display method of the "WINDOWS" method which is becoming mainstream in personal computers and the like is effective.

In order to make the secretary image realistic,
Three-dimensional (3D) image display is also effective. This 3D
In principle, the image display is such that separate images with different viewpoints are provided to the left and right eyes.

Specifically, the left and right images are displayed on the display screen 13 in a time-sharing manner, and glasses for taking in separate images for the left and right eyes using change glasses or liquid crystal shutter glasses, or the entire display screen 13 is used. A lenticular type with a semi-cylindrical lens, or a simple type such as a projection display in which a hollow is focused by a concave mirror is used.

It is not preferable to force the user to wear glasses in order to interact with the virtual secretary in the office, and the glasses-free method is preferable.

[0069]

As described in detail above, according to the image / speech response device of the present invention, the display means is a computer
Since the person image is displayed by graphics or a natural image, and the mouth portion of the person image displayed on the display means is moved according to the voice output from the voice response means, for example, by interacting with a human secretary. In an ambience-like atmosphere, the user can search for materials, etc. with only a few instructions given orally. As a result, it is possible to greatly save human machine operations in the office or the like.

By displaying the person image selected by the user from the person image storage means on the display means, the person image can be displayed according to the user's preference.

Further, the display of the person image on the display means is turned on when the user is detected by the user detecting means, and the display of the person image is turned off when the user is not detected. The device can be operated only when a user is present in the vicinity, and power consumption when not needed can be reduced.

Further, by turning on the display of the person image on the display means by the voice uttered by the user and turning off the display after a lapse of a predetermined time after the voice is interrupted, the operation can be stopped when it is unnecessary. Therefore, it is possible to reduce power consumption when unnecessary.

Further, by directing the display surface of the display means in the direction detected by the user detection means, the user does not have to be restricted from moving within the office, for example.

Further, the voice response means outputs the response contents by voice at the voice level corresponding to the detection result of the voice level detection means, so that the user can interact with the device at an appropriate voice level.

The voiceprint registration means in which the voiceprint pattern of the user is registered, the voiceprint analysis means for analyzing the voiceprint of the user, and the analysis result of the voiceprint analysis means match the voiceprint pattern in the voiceprint registration means. A response control for determining whether or not there is a match, and permitting the voice output by the voice response unit when it is determined to match by the determination unit, and prohibiting the voice output by the voice response unit when it is determined not to match. By providing the means, it becomes possible to maintain the confidentiality of information in the device.

Further, when the voiceprint pattern is registered in the voiceprint registration means, a registration voice output means for outputting a registration voice and a voiceprint pattern of the voice uttered by the user following the registration voice are registered in the voiceprint. By providing the registration control means for registering to the means, it becomes possible to easily and accurately register the voiceprint pattern of the user.

According to the image / speech response method of the present invention, the human image is displayed on the display means by computer graphics or a natural image, the language information included in the voice of the user is recognized, and the recognition result is obtained. At the same time that the response content corresponding to is output as a voice, the mouth portion of the person image displayed on the display means is moved according to the voice output, so that there is the same effect as the above invention.

[Brief description of drawings]

FIG. 1 is a block diagram showing a schematic configuration of an embodiment of an image / voice response device according to the present invention.

FIG. 2 is a conceptual diagram showing a state during operation of a virtual secretary in this embodiment.

[Explanation of symbols]

 1 CPU part 1A User detection sensor 2 Bus line 3 File part 4 Communication part 5 Operation panel 6 Audio input part 7 Audio output part 8 Video display part 11 User 13 Display screen

Continuation of the front page (51) Int.Cl. ⁶ Identification code Office reference number FI Technical display location G10L 3/00 RS 531 L C7.C8 571 H H04M 11/06

Claims (9)

[Claims] 1. A voice recognition unit for recognizing language information included in a user's voice, a voice response unit for outputting a response content corresponding to a recognition result of the voice recognition unit by voice, and displaying image / document information. An image / speech response device comprising display means, wherein a human image is displayed on the display means by computer graphics or a natural image, and the portion of the mouth of the human image displayed on the display means is converted into the voice. An image / speech response device characterized in that the image / speech response device is adapted to move in response to a sound output from a response means. 2. A person image storage means for storing a plurality of types of person images, and a person image selected by a user from these person images is displayed on the display means. The image / speech response device according to claim 1. 3. A user detection means for detecting the physical presence of a user near the apparatus, wherein when the user is detected by the user detection means, the image of the person on the display means is displayed. 3. The image / voice response apparatus according to claim 1, wherein the display is turned on and the display of the person image is turned off when the user is not detected. 4. The image according to claim 1, wherein the display of the person image on the display means is turned on by a voice uttered by a user, and the display is turned off after a lapse of a predetermined time after the voice is interrupted. -Voice response device. 5. The user detection means is configured to detect the direction in which the user is present, and the display surface of the display means is directed to the direction detected by the user detection means. The image / voice response device according to claim 3. 6. A voice level detecting means for detecting a voice level of a user, wherein the voice response means outputs the response contents by voice at a voice level according to a detection result of the voice level detecting means. The image / voice response device according to claim 1, wherein 7. A voiceprint registration unit in which a voiceprint pattern of a user is registered, a voiceprint analysis unit for analyzing a voiceprint of the user, and an analysis result of the voiceprint analysis unit matches a voiceprint pattern in the voiceprint registration unit. The determination unit that determines whether or not the determination unit and the determination unit determine that the voice response unit permits the voice output, and the determination unit determines that the voice response unit prohibits the voice output. The image / voice response device according to claim 1, further comprising a response control unit. 8. A registration voice output means for outputting a registration voice when the voiceprint pattern is registered in the voiceprint registration means, and a voiceprint pattern of a voice uttered by a user following the registration voice for the voiceprint registration. The image / speech response device according to claim 7, further comprising: registration control means for registering to the means. 9. A person image is displayed on a display means by computer graphics or a natural image, linguistic information included in a user's voice is recognized, and a response content corresponding to the recognition result is output at the same time. An image characterized by moving the mouth portion of the person image displayed on the display means in accordance with the audio output.
Voice response method.