JPH07140997A - Personal robot and its control method - Google Patents

Abstract

PURPOSE:To provide a robot device in which operations are controlled by using a voice synthesizing means and a voice detection means with a smaller scale circuit constitution. CONSTITUTION:The personal robot device is provided with a voice output block which is controlled by a microcomputer 114 in accordance with the program stored in a memory means 116 and performs voice synthesis by the waveform information stored in the memory and a voice input block 111 which judges the loudness of an external voice. The robot recognizes the user's voice only from the loudness of the voice. Since the communication between the user and the robot is conducted by voice communication, the robot device is controlled by a simple device constitution and the user controls the robot without using his hands.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a consumer-use personal robot apparatus, and more particularly, to an intelligent personal robot apparatus which recognizes a subject from a picked-up image and uses voice synthesis while learning.

[0002]

2. Description of the Related Art A conventional technique relating to subject extraction for image recognition is described, for example, in 1993 Television Society Annual Conference Proceedings, page 91, "A Study of Image Extraction Processing by Sequential Growth". As described above, there is known a technology of extracting a target object from the image pickup signal by using the extraction condition based on the color and the luminance of the target object after the image pickup signal converted from light to an electric signal is converted into a luminance signal and a color difference signal. Has been. Further, in a personal robot, Japanese Patent Laid-Open No.
A robot that is programmed by a personal computer and travels according to the program is known, as is seen in Japanese Patent Publication No. 82777/1987, “Hobby Robot for Hobby”.

[0003]

In making an intelligent personal robot using these techniques, it is desired to solve the following problems.

(1) In the user interface,
To realize a configuration in which the user can control the robot with his / her own voice without using hands with the simplest possible configuration.

(2) In order to reduce the number of times the user controls the robot as much as possible, let the robot act according to the user's request. However, it is difficult to specify the user's request at the stage of creating the robot, and the environment in which the robot is used cannot be specified.

(3) Electronic equipments such as computers and AV equipments are advancing day by day, and technological advances are so large that the equipments purchased will soon become old and if replaced by new ones, it is bad for the global environment.

(4) It is multitasking. Advanced robots take actions such as seeing, listening, and moving, but have the multitasking ability to perform them simultaneously rather than sequentially.

(5) Generally, the robot uses a battery as a power source, and the power consumption is reduced as much as possible to reduce the frequency of battery replacement.

[0009]

In order to solve the above-mentioned problem (1), the personal robot apparatus of the present invention generates a voice signal for asking the user for an action according to the situation where the robot is placed. Synthesizer, voice output means for outputting a voice signal generated by the voice synthesizer, voice input means for inputting an external voice, voice output by the voice output means, and then voice input by the voice input means And a means for determining the action of the robot according to the detection result of the volume detecting means.

In order to solve the above-mentioned problem (2), the personal robot apparatus of the present invention is an electrically rewritable non-volatile memory so that the robot gradually becomes smart according to the user's request and the environment in which it is used. Memory means is further provided and the data stored in the non-volatile memory means is sequentially updated in accordance with the determined detection result of the action sound volume detection means, whereby the robot gradually operates according to the user's request and the environment in which the robot is used. It was a learning type that makes you smart.

In order to solve the above-mentioned problem (3), the personal robot apparatus of the present invention is provided with a block board for each function so that these boards can be inserted and removed from the mother board, and only new parts are replaced or added. It has expandability that can be used.

In order to solve the above-mentioned problem (4), a dedicated calculation means is provided in the image processing means and the voice synthesis means in addition to the main calculation means.

In order to solve the above-mentioned problem (5), tactile sensing means or power saving means is provided, and when there is no command from the user, power other than the power required to detect the command is turned off.

[0014]

In the present invention, the "speech" can be emitted as needed by the voice synthesizing means. Furthermore, the "answer" to the "word" uttered by the robot can be detected as the "loud" and "small" information of the voice by the voice input means and the volume detection means, and the dialogue with the user can be made simple. It will be possible. If you use voice synthesis, you can utter quite complicated "words", so tentatively "answer" is "large".
Even if only "small" can be judged, it is possible to communicate at a fairly high level. Further, when a plurality of pairs of voice input means are provided, the direction in which the user is located can be known at the same time, so that more advanced judgments and operations are performed.

Further, in the present invention, since electrically rewritable nondestructive memory means (EEPROM etc.) is provided,
For example, the coefficient of the neural network can be stored in this, and the coefficient can be updated so as to better suit the user's intention from the result of the interaction with the user. This allows the user to gradually take actions that suit the user's intention while repeatedly interacting with the user, and the desired action can be performed without the user's frequent control.

Further, according to the present invention, since the boards can be inserted and removed for each block, the version can be easily upgraded by exchanging or adding a part of the boards.

Further, in the present invention, since the image recognizing means and the voice synthesizing means have a calculating means in addition to the main calculating means, they can simultaneously move, see, hear, and speak. If it is controlled by only one von Neumann type calculation means, it becomes an awkward robot because image recognition cannot be performed or movement cannot be changed while talking. However, if a plurality of calculation means are provided as in the present invention, it is possible to continue observing an image while talking, so that it is possible to cope with sudden changes, and at the same time perform neural network calculation. Because it is possible, it is efficient.

Further, in the present invention, the main power source is turned "on".
However, power consumption is avoided by turning off the power supplies other than the blocks necessary for detecting an instruction unless the user gives an instruction.

[0019]

Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is an embodiment showing the configuration of a robot apparatus according to the present invention. Of these, FIG. 1A is a front view of the robot apparatus of the present invention with the body cover 103 removed. As shown in the figure, the robot apparatus is configured based on the mother board 101. Then, the sensor block 110, the voice input block 111, the voice output block 112, the image input block 113, the microcomputer block 114, the non-destructive memory block 115, the external ROM block 116 are unitized for each function, and each of them is united. The block is connected to a connector 119 provided on the motherboard 101. Further, a control switch 102 (detailed view is FIG. 7) is attached to the mother board 101 to switch the operation mode of the robot. Furthermore, on the lower part of the motherboard 101, drive wheels 105R and L for running the robot device, and a drive device 106.
R and L are provided. Furthermore, at the bottom of the mother board 101, there is a power supply block 1 composed of a rechargeable battery such as a nickel-cadmium battery or a nickel-hydrogen battery.
Place 04. Next, the appearance of the robot apparatus will be further described with reference to FIGS. 1 (b) and 1 (c). Figure 1
(B) is a view of the body cover 103 of the robot apparatus viewed from above. As shown in the figure, an image pickup lens 117 is attached to the body cover 103 so that the optical axis thereof faces directly in front of the robot apparatus. And
A pair of microphones 109a and 109b are attached so as to be symmetrical with respect to the optical axis of the imaging lens 117. Further, one microphone 109c is attached toward the rear of the robot device. Then, the external ROM insertion opening 108 is aligned with the portion of the motherboard 101 where the external ROM is connected.
Is provided. FIG. 1C is a front view of the body cover 103 of the robot apparatus. As shown in the figure, an imaging lens 117 and a speaker 118 are arranged on the front surface of the body cover 103. Then, the touch sensor 107a is formed so as to sandwich them from the left and right,
107b is attached. As described above, the robot apparatus has a structure in which the body cover 103 is placed on the motherboard 101.

The control switch 102 switches the main power source of the robot apparatus between "ON" and "OFF",
The mode is switched between the external ROM mode and the communication mode. In FIG. 7, the changeover switch is of a dial type. Dial 7 to switch modes
Turn to align the "●" with the "▼" position. Explaining each mode, "ROM" is an external RO
M mode, "OFF" is the main power supply off, "ON" is the main power supply sound, and "LOAD" is the program load or data load mode.

Next, the structure and function of each block will be described. FIG. 2 is a block diagram showing an embodiment of the image input block 113. In the figure, the imaging lens 117
The light imaged on the image sensor 201 is converted by the image sensor into an image signal. The image pickup signal is subjected to gain adjustment by an automatic gain control amplifier (AGC) 202. The analog image pickup signal is A /
It is converted into a digital signal by the D converter 203 and supplied to the image extraction processing block 204. The image extraction processing block 204 performs a function of generating a luminance signal and a color difference signal by subjecting the image pickup signal to a matrix process, and extracts a subject to which the condition is satisfied by the color and the luminance specified by the image extraction microcomputer 205. Is written into the shape memory. Further, the image extraction processing block 204 has many functions such as an illuminance and color temperature detection function. Then, the image extraction microcomputer 205 reads out the data of the shape of the object written in the shape memory, calculates the object information such as color, position, area, and images such as illuminance, color temperature, luminance distribution, and color distribution. Get information about.

FIG. 3 is a block diagram showing an embodiment of the audio output block 112. In the figure, a voice synthesis microcomputer 301 is a microcomputer for operating the voice synthesis ROM 302 to generate words. Speech synthesis ROM 302
Is the voice data decomposed into vowels and consonants,
It is stored in the memory as waveform data representing each voice. To generate a voice, first, the head address in which the waveform data of phonemes (vowels and consonants) is written is transferred from the voice synthesis microcomputer 301 to the counter 308. The output of the counter 308 is input to the address input section of the voice synthesis ROM 302, and by counting up the counter 308, the phoneme waveform data is sequentially read from the voice synthesis ROM 302. Further, in the voice synthesis microcomputer 301, the start addresses of the consonants and vowels that make up each word are registered in advance for each word.
As a result, the counter addresses of the consonant and vowel start addresses of words prepared in advance in the voice synthesis microcomputer 301 are counted by the counter 30.
8 is transferred in order to synthesize the voice. Further, a RAM 303, an interpolation circuit 304, and an up / down counter 305 are provided in order to smoothly change the voice at the joint between the voices. Then, the interpolated synthesized voice signal is converted into an analog signal by the D / A converter 306. Then, the signal is amplified by the low band pass amplifier circuit 307 and output from the speaker 118. Here, the mechanism of interpolation is as follows. Now, when trying to switch the output signal of the audio output block from the output "a" to the output "b", first, "a" is stored in the RAM 303.
Signal is written, and voice synthesis ROM
RAM3 while outputting the signal "b" from 302
The signal "a" is read out from 03. At this time, if the output of the up / down counter 305 is "c", the output "y" of the interpolation circuit 304 is y = (1-c ÷ d) × a + c ÷ d If it is represented by × b, the output of the interpolation circuit 304 is set to “a”.
In order to smoothly change from "b" to "b", the output "c" of the up / down counter 305 may be counted up from "0" to "d".

The word data of the voice synthesis microcomputer 301 is also used.
A more natural speech synthesis is performed by providing each word of the data together with data for changing the up / down operation speed of the up / down counter 305.

Further, the pitch of the synthesized voice can be adjusted by changing the operating speed of the counter 308. To adjust the pitch of the sound, for example,
When synthesizing the voice "a", normally, if the head address of the signal "a" is transferred to the counter 308 and the voice is outputted from the voice synthesis ROM 302, "a" is given.
If the waveform data is read twice as fast as the normal signal "a" such as "a" and then voice synthesis is performed, a voice one octave higher than usual can be synthesized.

FIG. 4 is a block diagram showing an embodiment of the sensor block. The sensor block includes two touch sensors 107a and 107b, and the outputs of these two sensors are subjected to gain adjustment by amplifier circuits 4a and 4b. Each output is the microcomputer block 11
4 and is time-divisionally fetched from the A / D port.
Thus, when the robot device contacts an obstacle or a wall, the situation can be recognized.

FIG. 5 is a block diagram showing an embodiment of the voice input block 111. In the voice input block,
Three microphones 10 mounted as shown in FIG.
9a, 109b, and 109c convert surrounding sound waves into electric signals. Then, the detected audio signal is subjected to gain adjustment by AGC amplifiers 501a, 501b, 501c. Then, the gain-adjusted audio signal is rectified by the detection means 502a, 502b, 502c. The rectified audio signal is then passed through the low-pass filter 503a,
Only DC component is made by 503b and 503c. The audio signal of only this DC component is A of the microcomputer block 114.
It is transmitted to the / D port and processed in a time-sharing manner like the output of the sensor block. At this time, the microcomputer block 114 determines whether sound is present or absent depending on the input level from the audio block. Also, the arrival direction of the sound is determined from the difference between the input levels of the three microphones.

FIG. 6 shows the connection relationship between the microcomputer block 114 and other blocks. The microcomputer block 114 is provided with an I / O bus for data and addresses, to which an external ROM block 116 and a non-destructive memory block 115 are connected. Addresses are given to these two memories from an address bus common to the ROM incorporated in the microcomputer block 114, and different addresses are assigned to the respective addresses. Further, the microcomputer block 114 is provided with an input port for A / D conversion, into which the voice signal picked up by the voice input block 111 and the signal detected by the sensor block 110 are input. Then, the microcomputer block 114 processes this input signal in a time division manner. Furthermore, the microcomputer block 114 has an I / O port, and the traveling device control circuit 6, the voice output block 112, and the image input block 113 are connected to this I / O port. Then, the image information designated by the microcomputer block 114 is output from the image input block 113 and taken into the microcomputer block 114. And the voice output block 1
A control signal is transferred to 12 and the traveling device control circuit 6 in accordance with the result of each input of image, voice, and sensor. The non-destructive memory block 115 is a battery backup R
It is composed of a readable and writable non-volatile memory such as AM or EEPROM, and stores user programs and results learned by the robot apparatus. The external ROM block is a ROM in which game software, security software, etc. are pre-programmed, and the robot apparatus main body and the ROM are packaged so that they can be freely attached and detached. The user replaces this external ROM block according to the purpose of use of the robot. As a result, the user can operate and enjoy the robot apparatus according to the purpose without programming.

Next, a program example of the robot apparatus according to the present invention will be described. FIG. 8 is a flow chart showing an example of a program for observing and talking to an object. This program example will be described below using this flowchart. "Standby mode" after powering on and resetting hardware
The standby state (800) is reached. In the "standby mode", the voice input block 111 and the microcomputer block 114
Only the power is on, and the presence / absence of voice input is first checked (801). Therefore, if there is no voice input, the system waits until there is a voice input, and if there is a voice input, activates the voice output block 112 and confirms activation (802). Then, if the user confirms that the startup is not permitted as a result of the startup confirmation, it returns to the "standby mode" again,
If "Startup permission" is given, it will be in the "power on" state (80
3). In the "power on" state, the action is selected first. The action is selected using the result of calculation by the neural network (804). The result of the calculation may have a plus sign or a minus sign, and the action pattern to be selected is the maximum action pattern with a positive output among those without the "selection flag". When the action pattern is determined (805), a "selection flag" is set for the action pattern to confirm the action (808). If the result of the action confirmation is "permit", all the "selection flags" are canceled (809) so that the calculation result of the neural network which is the selected output of the action pattern selected for the input pattern increases. , The coefficient of the neural network is changed (810), and the action is executed (811). After performing an action, select the next action. If the result of the action confirmation is "non-permitted", the coefficient of the neural network is changed so that the selection output for the input pattern decreases (812). Then, the action is selected again (8
04). Also, as a result of the action selection, when the action pattern to be selected (a positive output and the selection flag is not sufficient) disappears, the "selection flag" of all the action patterns is canceled (806). , The action pattern of the minus sign in the calculation result for the input pattern at that time (the action pattern in which the selection flag did not stand)
The coefficient of the neural net is changed so that the selected output of (1) approaches zero (807). Then, again, it returns to the standby mode and waits for a sound to be input.

Next, a method of interacting with the robot device using the voice output device and the voice input device will be described. Figure 9
8 is a flow chart of a program of "confirm start-up" in FIG. The confirmation of the start is started by the fact that the sound is detected, and first, it is started by waiting until the sound becomes quiet (901). Then, when the sound stops, a control signal for outputting a word to the voice synthesis block is transferred to say "High, Oyobidesuka" (90
2). Then, if a sound is detected during a period of 2 seconds, it is determined that the activation is not permitted (903, 904), and if no sound is detected, the activation stop is confirmed. To confirm that the startup has been cancelled, say "Mouskoshi, Nemas" (90
5). Then, if no sound is detected within 2 seconds, it is determined to be "activation not permitted" (906, 907), and if a sound is detected, it is determined to be "activation permitted". In this program, when denying the words and actions of the robot device, it was decided to yell out "NO". FIG. 10 is a flowchart of the “confirm action” program in FIG. In the "confirmation of action", first, it is checked whether or not there is a sound input (1001), and if there is a sound, it waits until the sound stops. Then, when the sound stopped, say "○○ shimusu" (100
2) After that, if there is no sound for 2 seconds, it is determined that “action permitted” (1003, 1004). If a sound is detected within 2 seconds, it waits until the sound stops (1005). Then, when the sound stops, "Yamemasu" is uttered (1006). If no sound is detected during the two passages thereafter, it is determined that “action is not permitted” (1007, 10
08), if a sound is detected, it is judged that the stop has been denied, and the process waits until the return sound stops first. By programming the robot apparatus as described above, it is possible to make the robot apparatus confirm "permission" and "non-permission" of activation and action by the volume of the sound. As described above, even if the robot apparatus does not have a voice recognition function for recognizing the content of voice, the user can communicate with the robot by answering the question to the user of the robot with or without sound. It will be possible.

Next, as an action pattern program of the robot apparatus, a program for observing an object is shown in FIG.
It is shown in the flowchart of FIG. In observing an object, the direction of the object is first turned (1101). Then, it starts to move forward (11
02), the object is moved forward until either the screen becomes full (1104) or touches (1103). Then, after stopping (1105), the information obtained from the image input block is converted into words. Then, the word information of the word is sent to the voice output block, the recognition information is voice-synthesized, and the object is explained by outputting the voice from the voice output block in the word (1106).

FIG. 12 shows an example of a neural network used for action selection. The input of this neural network is that all the information (illuminance,
The color temperature, the size of the object, the movement of the object, the color of the object), the volume of the sound, the direction of arrival of the sound, the time (like daytime, nighttime, etc.), and the previous action. Also, these inputs are calculated using the coefficients stored in non-destructive memory. The calculation result is output to the final output after passing through several intermediate layers. These final outputs are numerical values showing the possibility of execution for each behavior pattern, and some have a plus sign and some have a minus sign. There are many other action patterns such as moving around, observing, talking, screen commentary, chasing, approaching, moving away, turning around, and the number of the action patterns and the number of outputs of the neural network are the same. ing. Further, the calculation by the neural network is sequentially performed by software using the coefficient held in the nondestructive memory. Moreover, each coefficient may be changed by feeding back the user's evaluation of the relationship between the input and the output.

[0032]

According to the present invention, first of all, in the personal robot apparatus, since the user and the robot apparatus can have a real voice conversation with a simple structure, the robot can be operated by the user without using the hand. It is possible to control the device.

Secondly, the non-destructive memory is frequently used by the user by updating and storing the calculation coefficient of the neural network for determining the action pattern so as to meet the user's intention from the result of the interaction with the user. Without control
The robot takes the desired behavior of the user.

Thirdly, since each function block has a board shape and can be freely attached to and detached from the mother board,
It is easy to upgrade and add new functions.

Fourth, since the calculation means dedicated to each functional block is used, the robot can simultaneously perform a plurality of tasks and can perform smooth movements.

Fifth, since the power supply is controlled so that only the minimum required function is operated depending on the situation, the power consumption can always be kept to the minimum.

Sixth, since the software is supplied from the external ROM according to the purpose, the robot apparatus can be operated for various purposes without the user having to program.

[Brief description of drawings]

FIG. 1 is an external view showing an example of the configuration of a robot apparatus.

FIG. 2 is a block diagram showing an embodiment of an image input block.

FIG. 3 is a block diagram showing an embodiment of an audio output block.

FIG. 4 is a block diagram showing an embodiment of a sensor block.

FIG. 5 is a block diagram showing an example of a voice input block.

FIG. 6 is a block diagram showing a connection relationship of each block.

FIG. 7 is a diagram showing an embodiment of a control switch.

FIG. 8 is a flowchart showing a program of a main program of the robot apparatus.

FIG. 9 is a flowchart showing a program for “confirmation of activation” of the robot apparatus.

FIG. 10 is a flowchart showing a program for “confirming behavior” of the robot apparatus.

FIG. 11 is a flowchart showing a “behavior” program of the robot apparatus.

FIG. 12 is a diagram showing an example of realizing action selection using a neural network.

[Explanation of symbols]

 101 Motherboard 108 External ROM Insertion 109a, b, c Microphone 110 Sensor Block 111 Audio Input Block 112 Audio Output Block 113 Image Input Block 114 Microcomputer Block 115 Nondestructive Memory Block 116 External ROM Block 117 Imaging Lens 118 Speaker

Claims (17)

[Claims] 1. A personal robot apparatus which stores a program for controlling an action in a memory means, and executes the program by a calculating means to perform an action according to the program, in a situation where the robot is placed. A voice synthesizing means for generating a voice signal in response to a user's action instruction, a voice output means for outputting the voice signal generated by the voice synthesizing means, a voice input means for inputting an external voice, and a voice outputting means. Volume detection means for detecting the volume of the voice input by the voice input means after outputting the voice,
And a means for determining the action of the robot according to the detection result of the volume detecting means. 2. The personal robot apparatus according to claim 1, wherein the voice detecting means detects the volume depending on whether the volume is higher or lower than a fixed value. 3. The personal robot apparatus according to claim 1, wherein a plurality of pairs of the voice input means and the volume detecting means are provided. 4. The personal robot apparatus according to claim 3, wherein there are three pairs of the voice input means and the volume detecting means. 5. Learning is further provided by further providing electrically rewritable non-volatile memory means and successively updating the data stored in the non-volatile memory means in accordance with the determined detection result of the action sound volume detection means. The personal robot apparatus according to claim 1. 6. The personal robot apparatus according to claim 1, further comprising a mother board capable of replacing a plurality of boards and mounting the boards according to the purpose of use of the robot. 7. The speech synthesizing means is controlled by a second memory means for storing a program different from the program and a second calculating means for executing the different program. Personal robot equipment. 8. The personal computer according to claim 1, further comprising a second memory means for storing a program different from the program, and an image processing means controlled by a second calculating means for executing the other program. Robot equipment 9. The personal robot apparatus according to claim 1, further comprising: tactile sensing means for detecting contact with an external object, and stopping traveling when the output of the tactile sensing means exceeds a predetermined value. 10. The personal robot apparatus according to claim 1, wherein said calculation means controls a plurality of power supplies in a plurality of "power-on" modes. 11. A means for receiving a command from a user is provided, and the calculating means sets a "power-on" mode in which the power consumption is minimum when there is no command, and stands by.
The personal robot device described. 12. A power on / off switch is provided,
11. The personal robot apparatus according to claim 10, wherein when the switch is in the "ON" state, the calculating means sets the "power-on" mode with the least power consumption. 13. A control method of a personal robot apparatus, wherein a program for controlling a behavior is stored in a memory means, and the behavior according to the program is controlled by executing the program by a computing means. Generates a voice that asks the user for an action instruction according to the circumstance, inputs an external voice, detects the volume of the input external voice, and determines the action of the robot from the detected volume. A method for controlling a personal robot apparatus, comprising: 14. The control method for a personal robot apparatus according to claim 13, wherein the sound volume is detected depending on whether the sound volume is higher or lower than a predetermined value. 15. The control method for a personal robot apparatus according to claim 13, wherein the voice is input from a plurality of points, and the sound volume is detected for each input from the plurality of points. 16. The method of controlling a personal robot apparatus according to claim 15, wherein the number of the voice input points is three. 17. The personal robot according to claim 13, wherein the data regarding the action determination is sequentially stored in an electrically rewritable nonvolatile memory, and the action determination is performed based on the stored data. Device control method.