JP2008254122A - Robot - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a robot capable of autonomously controlling behavior in accordance with the surrounding environment, situation of person and reaction from the person. <P>SOLUTION: The robot stores a map data which associates speech output information whether the robot can speak or not, and which indicates a volume of speech and tone of speech with position information, and a person situation data which associates the speech output information, an image condition, and a sound condition with each other. An approach action control means 48 is equipped with an environment information detection means 110 for detecting the speech output information corresponding to a present position from the map data, a person situation determination means 120 for determining the situation of a target person based on the image condition, a processed result of the photograph image, and a sound recognition result, and a response action control means 130 which extracts the speech output information corresponding to the situation of determined person from the person situation data, and decides whether the speech for the target person is possible or not, the volume of speech and the tone of speech based on the extracted speech output information and the speech output information detected by the environment information detection means 110. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a robot capable of interacting with a human, and more particularly to a robot capable of autonomous movement.

Conventionally, there is known a robot capable of interacting with a human by recognizing an input human voice and outputting predetermined information as a voice (see, for example, Patent Document 1 and Patent Document 2).
The robot described in Patent Document 1 is an interactive robot apparatus that actively and actively transmits information on the Internet to a user. This robot moves when it detects that the user is in an area where the user cannot interact, or when it detects that the user exists in an area where the user can interact but cannot detect the user's speech. Manipulate the site to perform non-reactive voluntary movements, draw the user's attention and induce the user to approach the robot. In addition, when the user utters, this robot gives priority to the user's utterance by stopping text output and browser operation.

The dialogue apparatus described in Patent Document 2 includes a robot that presents operation information stored in a database and a calculation processing result of a CPU by an operation, and outputs a sound at a timing determined based on the voice of the conversation partner and the body motion. To do. Specifically, the dialogue apparatus detects the delegation of the utterance right on the human side using the analysis result of the correlation between the voice and the body movement when the two people actually interact and the change of the utterance right, Speaking at that timing, we have a natural and smooth dialogue.
JP 2002-108388 A (paragraph 0014 to paragraph 0025, FIG. 1) Japanese Patent Laying-Open No. 2001-162573 (paragraphs 0009 to 0020, FIG. 1)

  However, since the robot described in Patent Document 1 or Patent Document 2 actively performs dialogue or actively transmits information on the Internet, it is different from the assumed usage environment or usage situation. The user may feel dissatisfied when speaking as expected. For example, when a device used in a noisy place is moved to a quiet place and used, the user must change the setting in order to lower the utterance volume of the device. In addition, when a device that has spoken in a friendly and friendly tone is diverted to business, the user must change the setting to make it a business tone. Therefore, there is a demand for a robot that can autonomously control the surrounding environment such as a place where the apparatus is actually used, the situation of the target person, and the behavior according to the reaction from the person.

  Therefore, an object of the present invention is to provide a robot that solves the above-described problems and can autonomously control the behavior according to the surrounding environment, the situation of the person, and the reaction from the person.

  The present invention was devised to achieve the above object, and the robot according to claim 1 of the present invention is a current position detecting means for detecting the current position of the robot on a preset map. And image processing means for performing image processing so that the situation of the person can be discriminated from a photographed image obtained by photographing the person to be communicated by the imaging means, and voice recognition and utterance so that the situation of the person can be distinguished from voice A robot having speech processing means to perform and approach behavior control means for controlling approach behavior to the target person before performing the utterance, and whether the utterance of the robot is set in advance, utterance volume Map data created by associating utterance output information indicating at least one of the utterance tone and position information indicating the position on the map; The approach behavior control means comprises storage means for storing utterance output information and human situation data created by associating preset image conditions indicating the situation of the person and voice conditions indicating a reaction from the person. An environment information detecting means for detecting speech output information corresponding to the detected current position from the map data as information resulting from the environment of the robot, the preset image condition, the processing result of the captured image, and Based on the result of the voice recognition, a human situation determination unit that determines the situation of the target person, and speech output information corresponding to the situation of the person determined by the human situation determination unit from the person situation data Utterances to the target person based on the extracted utterance output information and the utterance output information detected by the environment information detection means Availability, characterized in that it comprises a answering action control means for determining at least one of the speech sound and speech tone.

  According to such a configuration, the robot controls the approach behavior to the target person before speaking, and the speech output information detected corresponding to the current position by the response behavior control means, Based on the situation and the utterance output information extracted in response to the reaction from the person, at least one of utterance availability, utterance volume and utterance tone for the target person is determined. Therefore, the robot can change the availability of the utterance, the utterance volume, and the utterance tone according to the current position, the situation indicated by the currently captured image of the target person, and the current utterance situation of the person. Can do. Here, in the possibility of utterance, “impossibility of utterance” does not mean that utterance is not prohibited, but indicates that talking is not possible.

  The robot according to claim 2 is the robot according to claim 1, further comprising noise measuring means for measuring a noise around the robot and detecting a noise level, wherein the map data includes: For each preset noise level, the utterance output information is created in association with the position information, and the environment information detecting means generates utterance output information corresponding to the detected current position and the detected noise level. It is detected as information resulting from the environment of the robot from map data created for each preset noise level.

  According to such a configuration, the robot determines whether or not to speak depending on the current position, the noise measured at the location, the situation indicated by the currently captured image of the target person, and the reaction from the person, The utterance volume and utterance tone can be changed. That is, the utterance output information of the robot depending on the location is appropriately determined in consideration of the position information and noise. Therefore, for example, when a quiet place where the control for reducing the volume is to be performed on a daily basis is a noisy place that is used for a special purpose, the robot performs control for maintaining the volume as it is or Control to increase the volume can be performed.

  The robot according to claim 3 is the robot according to claim 1 or 2, wherein the response action control means extracts the utterance output information extracted from the person situation data and the environment information detection means. If the two are different from each other, and the two are different, each utterance output information is digitized and weighted to calculate an integrated value, and the calculated integrated value is set in advance. If the utterance volume level is smaller than the set value indicating the permission of the person, the behavior control unit for allowing the utterance to the person and the utterance volume level adjustment based on the integrated value when the utterance to the person is permitted or And an utterance level adjusting means for switching the utterance tone.

  According to such a configuration, in the robot, the utterance output information corresponding to the current location is different from the utterance output information corresponding to the situation indicated by the currently captured image of the target person and the reaction from the person. Even those integrated values are calculated. Then, the robot adjusts the level of the utterance volume or switches the utterance tone when utterance is permitted based on the calculated integrated value. Here, when the utterance is permitted, the robot can adjust the level of the utterance volume in consideration of the distance from the position where the robot actually utters to the position where the target person is located.

  According to a fourth aspect of the present invention, there is provided the robot according to the third aspect, wherein the robot drives at least one of a head, an arm, and a leg connected to the torso of the robot. An autonomous movement control means for autonomously moving the at least one part by outputting a drive signal to the drive means, and predesignating gestures that are physical actions for moving the at least one part when performing a predetermined utterance Scenario storage means for storing the created scenario, and gestures corresponding to utterances made to the target person are extracted from the scenario, and a command for specifying the extracted gestures is output to the autonomous movement control means When the speech volume level is adjusted by the gesture integration unit and the utterance level adjustment unit, the volume is adjusted in proportion to the adjusted utterance volume level. Wherein the by gestures specified further comprising a gesture adjusting means for adjusting the movement width of the site as a command.

  According to this configuration, when the level of the utterance volume is adjusted, the robot adjusts the movement width of each part by gesture in proportion to the adjusted level of the utterance volume. Therefore, the robot performs gestures with a relatively small movement width when the speech volume is relatively low, and performs gestures with a relatively large movement width when the speech volume is relatively high. Therefore, gestures during the utterance of the robot become natural. Here, the gesture includes an action of tilting the head, an action of raising and spreading the arm part, an action of moving the leg part up and down, etc. in order to display various intentions according to the utterance content.

  A robot according to a fifth aspect is the robot according to any one of the first to fourth aspects, wherein the image processing means extracts a face area of the target person from the captured image. A face recognizing means for extracting, and a line-of-sight detecting means for detecting a line-of-sight direction of the target person from the extracted face area, and after uttering a predetermined topic to the target person An interest level obtained by quantifying the gaze direction detected by the line-of-sight detection means is calculated, it is determined whether or not the calculated interest level has increased, and the person is interested in the topic when the interest level has increased. And an interest estimation unit that records the determination result, and a topic control unit that interrupts utterances related to the predetermined topic when the degree of interest decreases. To do.

  According to such a configuration, the robot can utter an utterance related to a predetermined topic to the target person, and then calculate the degree of interest from the gaze direction of the person and acquire information on the topic whose degree of interest has increased. . In general, when a person is interested in what the speaker speaks, the gaze direction is directed toward the speaker. Will reflect the person's interest in Therefore, the calculated interest level reflects the interest of the person. In addition, when the degree of interest decreases, specifically, when a person turns away the line of sight from the robot, the robot stops providing the topic being spoken. Therefore, a person who is provided with a topic from the robot does not have to keep listening to a topic that is not interested in, and thus it is easy to feel familiar with the robot.

  Moreover, the robot according to claim 6 is the robot according to any one of claims 1 to 4, wherein the sound processing means is configured to input a sound volume of the target person from an input sound, Speech information detecting means for detecting person speech information indicating at least one of sound quality and speech speed, and speech recognition by the speech processing means after starting speech on a predetermined topic for the target person When the degree of interest is calculated by calculating the degree of interest obtained by converting the recognition result obtained or the person utterance information detected by the utterance information detecting unit into a numerical value, and determining whether the calculated degree of interest has increased. An interest estimation unit that determines that the person is interested in the topic and records the determination result; a topic control unit that interrupts an utterance related to the predetermined topic when the degree of interest decreases; The Characterized in that it comprises the al.

  According to such a configuration, the robot performs an utterance on a predetermined topic to a target person, calculates an interest level from the person utterance information detected from the voice of the person, and information on the topic whose degree of interest has increased. Can be obtained. When a person loses interest in what the other party is talking to, the volume of the reply voice returned to the other party becomes lower, the sound quality is lower, or the speaking speed is slower. The person utterance information detected later reflects the person's interest regarding the topic that has been voiced. Therefore, the calculated interest level reflects the interest of the person. In addition, when the interest level decreases, specifically, when the volume of the answering voice of the person decreases, the sound quality decreases, or the speaking speed slows down, the robot selects the topic being spoken. Suspend providing. Therefore, a person who is provided with a topic from the robot does not have to keep listening to a topic that is not interested in, and thus it is easy to feel familiar with the robot.

  The robot according to claim 7 is the robot according to claim 5 or 6, further comprising personal information storage means for storing a plurality of topics, wherein the topic control means determines the degree of interest. Based on this, the topic stored in the person information storage means is switched, and the utterance related to the switched topic is made to the target person.

  According to this configuration, the robot stores a plurality of topics, and switches the topics to be provided according to the change in the degree of interest calculated from the information acquired after speaking to the target person. Therefore, the robot can estimate a topic preferred by the conversation partner based on the degree of interest during the conversation and speak.

According to the first aspect of the present invention, the robot can autonomously control the behavior according to the surrounding environment and the situation of the person.
According to the second aspect of the present invention, the robot can speak in a different manner depending on noise even at the same place.
According to the invention described in claim 3, the robot can integrate the action determined according to the surrounding environment and the action determined according to the situation of the person.
According to the fourth aspect of the present invention, the robot can perform natural gestures against changes in the utterance volume.

According to the invention described in claim 5 or 6, the robot can quantitatively acquire information related to the interest of the conversation partner.
According to the seventh aspect of the present invention, since the robot utters by estimating the topic preferred by the conversation partner, the person who is the conversation partner can easily feel the friendliness and friendliness of the robot. Further, the robot can effectively collect information such as the preference of the person by switching the topic and continuing the conversation with the person.

  Hereinafter, the best mode for carrying out the robot of the present invention (hereinafter referred to as “embodiment”) will be described in detail with reference to the drawings. First, an overall configuration of a robot control system A including a robot according to an embodiment of the present invention will be described with reference to FIG. FIG. 1 is a diagram schematically showing a configuration of a robot control system including a robot according to an embodiment of the present invention.

(Configuration of robot control system A)
As shown in FIG. 1, a robot control system A includes a robot R, a base station 1 connected to the robot R by wireless communication, and a management computer connected to the base station 1 via a robot dedicated network 2. 3 and a terminal 5 connected to the management computer 3 via a network 4.

As shown in FIG. 1, the robot control system A has a plurality of robots R _A , R _B , and R _C having a moving function (however, when a robot is not specified, it is simply referred to as a robot R). Each robot R executes a task in accordance with a task execution plan (task schedule) preset for each robot R in the management computer 3.

Here, an autonomous mobile biped robot will be described as an example.
The robot R executes a task in accordance with an execution command input from the management computer 3, and at least one robot R is arranged in a task execution area set in advance as an area where the robot R executes the task.
Here, FIG. 1 shows a robot _RA that is executing a task (guidance task) that guides a visitor to a predetermined place such as a conference room, and a task (package) that delivers a package to a person. A robot R _B executing the delivery task) and a robot R _C waiting until a new task is assigned are illustrated.

  As shown in FIG. 2, the robot R has a head R1, an arm R2, a leg R3, a torso R4, and a rear housing R5, and the head R1 and arms connected to the torso R4, respectively. The part R2 and the leg part R3 are each driven by an actuator (driving means), and bipedal walking is controlled by the autonomous movement control part 50 (see FIG. 6). Details of this bipedal walking are disclosed in, for example, Japanese Patent Application Laid-Open No. 2001-62760.

  For example, when executing a guidance task, the robot R guides the person H in a predetermined guidance area (a movement area such as an office or a corridor). Here, the robot R detects whether or not the person H having the tag T exists in the peripheral area by transmitting light (for example, infrared light, ultraviolet light, laser light, etc.) and radio waves to the surroundings. Then, the position of the detected person H is specified and approached, and based on the tag T, personal identification as to who the person H is is performed. The tag T receives infrared light and radio waves emitted from the robot R to specify the position (distance and direction) of the person. The tag T generates a reception report signal including a tag identification number based on the signal indicating the light receiving direction included in the received infrared light and the robot ID included in the received radio wave, and returns it to the robot R. To do. The robot R that has received the reception report signal can recognize the distance and direction to the person H wearing the tag T based on the reception report signal, and can approach the person H.

  When the robot R autonomously moves within the guidance area to execute a certain task (for example, a guidance task or a package delivery task), the robot R irradiates laser slit light or infrared rays to search for a road surface state or a mark on the road surface. It is supposed to be. In other words, the robot R knows where the robot is moving in the moving area, and when in the normal moving area, the robot R irradiates the road surface with laser slit light to check for road steps, swells, and obstacles. When the mark M is within the installation area of the mark M, the mark M is detected by irradiating the road surface with infrared rays, and the self position is confirmed and corrected. Here, the mark M is a member made of a reflective material that recursively reflects infrared rays, for example. The mark M has position data, and the position data is stored in the storage unit 30 (see FIG. 6) in a form included in the map data. The map data includes the position data of the mark M installed at a specific location in the guidance area and the data related to the installation area of the mark M having a predetermined width (range) in the position data. . The mark M installation area refers to an area within a predetermined distance from the mark M. For example, a circular area having a radius of 1 to 3 m around the mark M, or 3 m before the mark M (robot side). It is arbitrarily set like a rectangular area.

Returning to FIG. 1, the description of the configuration of the robot control system A will be continued.
The base station 1 mediates data exchange between the robot R and the management computer 3.
Specifically, the base station 1 transmits an execution command output from the management computer 3 to the robot R, and also transmits data (status information) on the state of the robot R transmitted from the robot R and the execution command from the robot R. Is received and a signal (reception report signal) is received and output to the management computer 3.
At least one base station 1 is provided in the task execution area in order to ensure data exchange between the robot R and the management computer 3.
In addition, when the task execution area is set over several floors of the building, it is preferable to be provided for each floor. When one base station 1 cannot cover all the task execution areas, A plurality of base stations 1 are preferably provided in the task execution area.

  The robot dedicated network 2 connects the base station 1, the management computer 3, and the network 4, and is realized by a LAN (Local Area Network) or the like.

  The management computer 3 manages a plurality of robots R, and performs various controls such as movement and speech of the robot R via the base station 1 and the robot dedicated network 2 and information necessary for the robot R. I will provide a. Here, the necessary information corresponds to the name of the detected person, a map around the robot R (local map), and the like. These pieces of information are stored in the storage unit 3a of the management computer 3. ing.

  FIG. 3 is a diagram showing an example of a local map used in the robot system shown in FIG. Here, as shown in FIG. 3A, the guide area 301 is a rectangular area on the floor where the building is located. The robot R or a person to be guided by the robot R enters the guide area 301 through the passage 303 outside the entrance / exit 302 of the guide area 301. Inside the entrance / exit 302, a hall 304 spreads out, a reception 305 is arranged at the back corner of the hall 304, and a security room 306, a common room 307, and a conference room partitioned as a private room on the wall side of the guide area 301. Each chamber 308 is provided. The reception 305 includes an L-shaped counter table 305a and a counter space 305b in which reception staff are arranged. The base station 1 is installed in the counter space 305b. As shown in FIG. 3B, the noise level detected by the robot R varies depending on the location of the guide area 301. In the time zone when the robot R performs the guidance task, the noise level in the hall 304 is, for example, “60 dB”. Similarly, the noise levels of the security room 306, the common room 307, and the conference room 308 are, for example, “70 dB”, “100 dB”, and “50 dB”, respectively. The management computer 3 includes a local map (local map data) in which information on local maps such as passages and rooms is registered in association with position coordinate data, and a global map that is map information of a task execution area in which the local maps are accumulated. Are stored in the storage unit 3a (see FIG. 1).

Further, the management computer 3 holds a task information database that stores information (task data) related to tasks to be executed by the robot R in the storage unit 3a (see FIG. 1).
As shown in FIG. 4, the task information database 400 includes a task ID that is a unique identifier assigned to each task, a task priority, a task importance, a robot ID that is an identifier of a robot that executes the task, Contents of tasks such as guidance and transportation (package delivery), the position where the task starts within the task execution area (start position), the position where the task ends within the task execution area (end position), and the time required to execute the task ( Time required), scheduled task start time (start time), scheduled task end time (end time), task status, and the like are included as information items.

The management computer 3 sets an execution plan (task schedule) for tasks to be executed by the robot R for each robot R.
As shown in FIG. 5, the task schedule table 500 includes an execution order of tasks to be executed by the robot R, a task ID for identifying a task registered in the task information database 400 (see FIG. 4), a task priority, It is a table that includes task contents and task status as information items.
In this task schedule table 500, these information items are arranged for each robot R arranged in the task execution area, and what kind of tasks are assigned to each robot R in what order. It is possible to grasp.

Returning to FIG. 1 again, the description of the configuration of the robot control system A will be continued.
The terminal 5 is connected to the management computer 3 via the network 4 and registers information related to a person in the storage unit 3a of the management computer 3 or corrects the registered information. The terminal 5 is used for registering tasks to be executed by the robot R, changing a task schedule set in the management computer 3, inputting an operation command for the robot R, and the like.

  Hereinafter, the robot R will be described in detail.

[robot]
As shown in FIG. 6, in addition to the head R1, the arm R2, the leg R3, the trunk R4, and the rear housing R5, the robot R includes cameras C and C, speakers at appropriate positions of these parts R1 to R5. S, microphone MC, MC, image processing unit 10, audio processing unit 20, storage unit 30, main control unit 40, autonomous movement control unit 50, wireless communication unit 60, battery 70, object detection unit 80, and surrounding state detection unit 90.
Furthermore, the robot R includes a gyro sensor SR1 that detects the direction in which the robot R is facing, and a GPS (Global Positioning System) receiver SR2 that acquires the position coordinates of the robot R on a preset map. Have.

[camera]
Cameras (photographing means) C and C are capable of capturing an image of the robot R in the forward movement direction as digital data. For example, a color CCD (Charge-Coupled Device) camera is used. The cameras C and C are arranged side by side in parallel on the left and right, and the captured image is output to the image processing unit 10. The cameras C and C, the speaker S, and the microphones MC and MC are all disposed inside the head R1. The speaker (sound output means) S can emit a predetermined sound synthesized by the sound processing unit 20.

[Image processing unit]
The image processing unit (image processing means) 10 processes images (captured images) taken by the cameras C and C, and grasps the situation around the robot R from the taken images. It is the part which recognizes. The image processing unit 10 includes a stereo processing unit 11a, a moving body extraction unit 11b, a face recognition unit 11c, and a line-of-sight detection unit 11d.
The stereo processing unit 11a performs pattern matching on the basis of one of the two images taken by the left and right cameras C and C, calculates the parallax of each corresponding pixel in the left and right images, and generates a parallax image. The generated parallax image and the original image are output to the moving object extraction unit 11b. This parallax represents the distance from the robot R to the photographed object.

The moving body extraction unit 11b extracts a moving body in the photographed image based on the data output from the stereo processing unit 11a. The reason for extracting the moving object (moving body) is to recognize the person by estimating that the moving object is a person.
In order to extract the moving object, the moving object extraction unit 11b stores images of several past frames (frames), compares the newest frame (image) with the past frames (images), and Pattern matching is performed, the movement amount of each pixel is calculated, and a movement amount image is generated. Then, from the parallax image and the movement amount image, when there is a pixel with a large movement amount within a predetermined distance range from the cameras C and C, it is estimated that there is a person, and as a parallax image of only the predetermined distance range The moving body is extracted, and an image of the moving body is output to the face recognition unit 11c.

The face recognition unit (face recognition means) 11c recognizes the face area and the face position from the size, shape, etc. of a part of the extracted moving body. Similarly, the position of the hand is also recognized from the size and shape of a part of the extracted moving body.
The recognized face position is output to the main control unit 40 and also to the line-of-sight detection unit 11d as information when the robot R moves and to communicate with the person.

The line-of-sight detection unit (line-of-sight detection means) 11d detects the line-of-sight direction of the person to be recognized from the face area extracted by the face recognition unit 11c.
The line-of-sight detection unit 11d analyzes the image around the eyes to determine whether the eyes are closed, detects the pupil when the eyes are not closed, and determines the line-of-sight direction from the detected pupil position and eyeball position. To detect. For example, the line-of-sight direction is obtained from the relationship between the position and posture of the face recognized by the face recognition unit 11c and the center position of the pupil. In this case, the line-of-sight direction detected by the line-of-sight detection unit 11d is obtained as a vector connecting the center position of the eyeball and the center position of the pupil. The obtained gaze direction is output to the main control unit 40 for use in calculating the degree of interest.

[Audio processor]
The speech processing unit 20 includes a speech synthesis unit 21a, a speech recognition unit 21b, a sound source localization unit 21c, a noise measurement unit 21d, and an utterance information detection unit 21e.
The voice synthesizer 21a is a part that generates voice data from character information (text data) and outputs the voice to the speaker S based on the speech action command determined and output by the main controller 40. For the generation of the voice data, the correspondence between the character information (text data) stored in the storage unit 30 in advance and the voice data is used. The audio data is acquired from the management computer 3 and stored in the storage unit 30.
The voice recognition unit (voice recognition unit) 21 b receives voice data from the microphones MC and MC, generates character information (text data) from the input voice data, and outputs the character information to the main control unit 40. The voice recognition unit 21b performs voice recognition so that the situation of a person can be determined from the voice. For example, by recognizing a voice such as “Yes” or “What?”, The main control unit 40 can determine that “there is a reaction from a person”. The correspondence relationship between the voice data and the character information (text data) is stored in the storage unit 30 in advance.
The sound source localization unit 21 c specifies a sound source position (a planar position recognized by the robot R) based on the sound pressure difference between the microphones MC and MC and the sound arrival time difference, and outputs the sound source position to the main control unit 40. The sound source position is represented by, for example, a rotation angle θ _z around the direction in which the robot R stands (z-axis direction).

The noise measuring unit (noise measuring means) 21d measures the noise around the robot R and detects the noise level. The detected noise level is output to the main control unit 40 for use in adjusting the speech volume.
The utterance information detection unit (speech information detection) 21e detects person utterance information related to the voice of the person to be recognized from the voices input from the microphones MC and MC. The person utterance information indicates at least one of the volume, sound quality, and speech speed of the person's voice. The detected person utterance information is output to the main control unit 40 for use in calculating the degree of interest.

[Memory]
The storage unit (storage means) 30 is composed of, for example, a general hard disk or the like, and stores necessary information (local map data, conversation data, etc.) transmitted from the management computer 3. In this embodiment, normal tone data uttered for normal use and special tone data uttered for special purposes are stored as conversation data. Here, the normal tone is, for example, a spoken tone used by a person in an everyday scene, and the special tone is, for example, an honorific tone used by a person in a business or ritual scene.
Further, the storage unit 30 stores information necessary for performing various operations of the main control unit 40, as will be described later.

[Main control section]
The main control unit 40 controls the image processing unit 10, the sound processing unit 20, the storage unit 30, the autonomous movement control unit 50, the wireless communication unit 60, the target detection unit 80, and the surrounding state detection unit 90. The data detected by the gyro sensor SR1 and the GPS receiver SR2 is output to the main control unit 40 and used to determine the behavior of the robot R. The main control unit 40 includes, for example, control for communicating with the management computer 3, control for executing a predetermined task based on a task execution command acquired from the management computer 3, and the robot R as a destination. In order to perform control for movement, control for identifying a person, and control for interacting with a person, various determinations are made and commands for the operation of each unit are generated.

[Autonomous Movement Control Unit]
The autonomous movement control unit 50 drives the head R1, the arm R2, and the leg R3 in accordance with instructions from the main control unit 40. Although not shown, the autonomous movement control unit 50 includes a head control unit that drives the head R1, an arm control unit that drives the arm R2, and a leg control unit that drives the leg R3. These head control unit, arm control unit, and leg control unit output drive signals to actuators that drive the head R1, arm R2, and leg R3. This autonomous movement control part 50 and leg part R3 comprise a moving means.

[Wireless communication part]
The wireless communication unit 60 is a communication device that transmits and receives data to and from the management computer 3. The wireless communication unit 60 includes a public line communication device 61a and a wireless communication device 61b.
The public line communication device 61a is a wireless communication means using a public line such as a mobile phone line or a PHS (Personal Handyphone System) line. On the other hand, the wireless communication device 61b is a wireless communication unit using short-range wireless communication such as a wireless LAN conforming to the IEEE802.11b standard.
The wireless communication unit 60 performs data communication with the management computer 3 by selecting the public line communication device 61 a or the wireless communication device 61 b in accordance with a connection request from the management computer 3.

  The battery 70 is a power supply source necessary for the operation and processing of each unit of the robot R. The battery 70 has a rechargeable configuration and is replenished with electric power in a battery replenishment area (see FIG. 1).

[Target detection unit]
The target detection unit (target detection means) 80 detects whether or not there is a person with the tag T around the robot R. The target detection unit 80 includes a plurality of light emitting units 81 (only one is displayed in FIG. 6). These light emitting units 81 are constituted by LEDs, for example, and are arranged on the front and rear, right and left along the outer periphery of the head R1 of the robot R (not shown). The target detection unit 80 transmits infrared light including a signal indicating a light emitting unit ID for identifying each light emitting unit 81 from the light emitting unit 81 and receives a reception report signal from the tag T that has received the infrared light. To do. The tag T that has received any infrared light generates a reception report signal based on the light emitting unit ID included in the infrared light, so that the robot R determines the light emitting unit ID included in the reception report signal. By referencing, it is possible to specify in which direction the tag T exists as viewed from the robot R. Further, the target detection unit 80 has a function of specifying the distance to the tag T based on the radio wave intensity of the reception report signal acquired from the tag T. Therefore, the target detection unit 80 can specify the position (distance and direction) of the tag T as the position of the person based on the reception report signal. Further, the target detection unit 80 not only emits infrared light from the light emitting unit 81 but also transmits a radio wave including a signal indicating the robot ID from an antenna (not shown). Thus, the tag T that has received the radio wave can correctly identify the robot R that has transmitted infrared light. Details of the target detection unit 80 and the tag T are disclosed in, for example, Japanese Patent Application Laid-Open No. 2006-192563.

[Ambient condition detector]
The peripheral state detection unit 90 detects the peripheral state of the robot R, and can acquire self-position data detected by the gyro sensor SR1 and the GPS receiver SR2. The peripheral state detection unit 90 includes a laser irradiation unit 91 that irradiates slit light toward the search region, an infrared irradiation unit 92 that irradiates infrared light toward the search region, and a search region irradiated with slit light or infrared rays. And a floor camera 93. The peripheral state detection unit 90 detects a road surface state by analyzing a slit light image (an image when the slit light is irradiated) captured by the floor camera 93. Further, the peripheral state detection unit 90 analyzes the infrared image captured by the floor camera 93 (image when irradiated with infrared rays) to detect the mark M (see FIG. 2), and the position of the detected mark M The relative positional relationship between the mark M and the robot R is calculated from (coordinates). Details of the peripheral state detection unit 90 are disclosed in, for example, Japanese Patent Application Laid-Open No. 2006-167844.

[Configuration of main controller]
FIG. 7 is a block diagram showing the configuration of the main control unit of the robot shown in FIG.
The main control unit 40 includes a stationary obstacle integration unit 41, an object data integration unit 42, an action pattern unit 43, a gesture integration unit 44, an internal state detection unit 45, an action plan management unit 46, and a motivation management unit. 47.

  The stationary obstacle integration unit 41 integrates information related to the peripheral state of the robot R detected by the peripheral state detection unit 90 and outputs the information to the behavior pattern unit 43. For example, when the stationary obstacle integration unit 41 detects an obstacle such as a cardboard box on the floor surface of the path of the robot R or detects a step on the floor surface, the behavior pattern unit 43 is integrated. Based on the obstacle information, a bypass route is searched by a local avoidance module (not shown).

  The object data integration unit 42 integrates identification data (object data) related to the object (object) based on the posture data of the robot R, the input data from the image processing unit 10, the target detection unit 80, and the sound source localization unit 21c. The integrated object data is output to the object data storage means 31 of the storage unit 30. As a result, an object map, which is data in which object data is recorded for each object and for each time, is generated in the object data storage unit 31.

[Composition of object map]
Here, the configuration of the object map stored in the object data storage means 31 will be described with reference to FIG. FIG. 8 is a diagram illustrating an example of object data.
The object map includes a plurality of time-specific data 801 classified by time. Each time data 801 includes a count 802 as time information, posture data, a camera posture and a noise level, and a table 803. The posture data is represented by, for example, a face position (x, y, z) and a face direction (θx, θy, θz), and the camera posture is, for example, a rotation angle (pan, tilt, pan, tilt, roll axis) (tilt, role). The noise level is detected by the noise measurement unit 21d (see FIG. 6) and is expressed in decibels (dB). In this table 803, a target (object) to be identified is arranged in a column, a plurality of items characterizing this object are arranged in a row, and a record is accumulated for each object (for each column). Details of each item will be described below.

The object number 804 is assigned up to N in the order in which the robot R detects the objects. In this table 803, eleven (N = 11) objects “0” to “10” can be managed. It is like that.
The body position 805 is position coordinate data output from the image processing unit 10 and is represented by the gravity center position coordinates (x, y) of the person (object) on the coordinate plane recognized by the robot R.
The speed 806 is speed data output from the image processing unit 10 and is represented by the moving speed (Vx, Vy) of the person (object) on the coordinate plane recognized by the robot R.

The person ID 807 is an identification number for identifying a person.
The person accuracy 808 indicates the accuracy of the person ID 807, and is defined as 100% perfect match.
The person life count 809 represents the elapsed time on the object data of the data registered in the person ID 807.

The RFID identification number 810 is an identification number of a person (object) recorded on the tag and is output from the target detection unit 80.
The RFID position 811 is position data output from the target detection unit 80, and is represented by an area determined by the distance and direction to the tag (object) around the robot R.
The RFID accuracy 812 indicates the accuracy of the data (identification number) of the RFID identification number 810.
The RFID life count 813 represents an elapsed time on the object map of data (identification number) registered in the RFID identification number 810.

The sound source position 814 is data output from the sound source localization unit 21c, and is represented by an angle θz of a person (object) speaking on a coordinate plane recognized by the robot R.
The sound source accuracy 815 indicates the accuracy of the data of the sound source position 814.
The sound source life count 816 represents the elapsed time on the object map of the data (position coordinates) registered at the sound source position 814.

The object life count 817 represents a count started when any one of person data, RFID identification data, and sound source identification data is input to an object for the first time.
TOTAL_ID 818 is an object identification number determined by the object data integration unit 42 based on the person ID 807 and the RFID identification number 810.
The TOTAL_accuracy 819 indicates the accuracy of the object identification number determined by the object data integration unit 42 based on the person accuracy 808 and the RFID accuracy 812.

The description of the configuration of the main control unit 40 will be continued with reference to FIG.
As will be described later, the behavior pattern unit 43 includes approach behavior control means 48 that controls approach behavior to a person before speaking to the person in accordance with a predetermined behavior (behavior pattern) of the robot R. Yes.
The behavior pattern unit 43 stores various programs (modules) for executing the behavior pattern, and reflects the behavior pattern by referring to the storage unit 30 when the behavior pattern is executed. .

  In the present embodiment, as shown in FIG. 7, in addition to the object data storage unit 31, the local map data storage unit 32, the motivation index storage unit 33, the scenario storage unit 34, and the personal information storage are stored in the storage unit 30. Means 35. Note that the storage unit 30 also stores position information of a predetermined person. Here, the position information of the person is information indicating the location of the person, and is created in advance in association with the day of the week, time, or the like, for example.

The local map data storage means 32 stores a map (local map) around the robot R described with reference to FIG. This local map is acquired from the management computer 3, for example.
The motivation index storage means 33 stores a motivation index. Since the motivation index is managed by the motivation management unit 47, details will be described later.

The scenario storage unit 34 stores scenarios (scripts) corresponding to various behavior patterns. The scenario relates to an operation such as stopping a person or an obstacle (object) while walking, 1m before the object, and raising the arm R2 to a predetermined position 10 seconds after stopping. There is something about utterance.
In addition, the scenario storage unit 34 stores a scenario created in advance for designating a gesture that is a body motion that moves at least one of the head R1, the arm R2, and the leg R3 when a predetermined utterance is performed. Remember.

  The person information storage means 35 stores a plurality of topics that a predetermined person likes. In the present embodiment, the person information storage means 35 stores topics by person and by genre. The topic stored in the person information storage means 35 is used for the speech behavior of the robot R.

  The behavior pattern unit 43 is a module that executes behavior patterns according to various scenes and situations by appropriately using the object data storage unit 31, the local map data storage unit 32, the scenario storage unit 34, and the person information storage unit 35. I have. Examples of the module include a destination movement module, a local avoidance module, a delivery module, a guidance module, a human correspondence module, and the like.

The destination movement module performs route search (for example, search for a route between nodes) and movement from the current position of the robot R to a destination such as a task execution position in the task execution area. This destination movement module obtains the shortest distance to the destination while referring to the map data and the current position.
The local avoidance module searches for a detour route for avoiding an obstacle based on the obstacle information integrated by the stationary obstacle integration unit 41 when an obstacle is detected during walking.

The delivery module executes an operation of receiving (grabbing) an article from a person who requests transportation of the article (client) and an operation of delivering the received article to the recipient (releasing the article).
The guidance module executes, for example, a task for guiding a visitor who has visited a guidance start point in the task execution area to a reception staff in the reception 305 of the guidance area 301 (see FIG. 3).
The person handling module performs, for example, speech, posture change, vertical movement and gripping of the arm portion R2 based on a predetermined scenario when executing an article transport task or a guidance task.

  The gesture integration unit (gesture integration unit) 44 extracts gestures corresponding to the utterances made to the target person from the scenario storage unit 34, and outputs a command for specifying the extracted gestures to the autonomous movement control unit 50. It is. Gesture by the movement of the head R1, for example, tilting the head R1 downward to display “bowing”, “thanks”, “consent”, “apology”, etc., or tilting the head R1 left and right This includes an action that conveys the intention expression “I don't know”. The gesture by the movement of the arm part R2 is, for example, an action of displaying “joy”, “praise”, etc. by raising the arm part R2, expanding the arm part R2 to the left and right, or shaking hands. This includes an action to convey an expression of “welcome”. The gesture by the operation of the leg portion R3 includes, for example, an operation of transmitting intention indications such as “joy” and “goodness” by running on the spot.

  The internal state detection unit 45 detects the internal state of the robot R and outputs the detection result to the motivation management unit 47. In the present embodiment, the internal state detection unit 45 detects the remaining amount of the battery 70. The detected remaining battery level is output to the motivation management unit 47. In addition, the internal state detection unit 45 generates data regarding the state of the robot R (current position, remaining battery level, task execution status, etc.) as status information at predetermined time intervals, and the generated status information is transmitted to the wireless communication unit 60. To the management computer 3. Then, the management computer 3 registers the input status information for each robot R in a robot information database (not shown) stored in the storage unit 3a.

  The action plan management unit 46 manages an action plan for executing various modules included in the action pattern unit 43 according to a predetermined schedule. In the present embodiment, the action plan management unit 46 manages an action plan for executing a predetermined task based on a task execution command acquired from the management computer 3, and is a module necessary for work to be currently executed. Is appropriately selected. The action plan management unit 46 appropriately selects a module necessary for the action plan for the identification target based on an instruction from the motivation management unit 47.

  The motivation management unit 47 manages the motivation index, and when the battery replenishment is not necessary and there is no task to be executed at present, an action plan for actively acting based on the motivation index The addition is instructed to the action plan management unit 46. Here, the motivation index indicates the magnitude of the feasibility of future action on a known predetermined object. In the present embodiment, the object is a human. The motivation index is determined based on the elapsed time index related to the time elapsed from the latest action of the robot R to the human and the action index related to the number of actions and the action time of the robot R with respect to the past target person.

  Specifically, the elapsed time index is set to increase in proportion to the amount of time that has elapsed since the action of the robot R on the latest target person, and the action index is the action of the robot R on the past target person. Is set so as to increase in proportion to the number of times and the amount of action time. That is, when the robot R is anthropomorphic, a large elapsed time index means that the person has a high degree of nostalgia, and a large action index means that the person has a high degree of intimacy. By setting in this way, the motivation index can be used to select a highly retrospective person or a highly intimate person from a plurality of known persons. Since the behavior index represents intimacy, hereinafter, the behavior index is referred to as an emotional index.

  The motivation index managed by the motivation management unit 47 is stored in the motivation index storage means 33. In the present embodiment, the motivation index storage means 33 stores a motivation index table. As shown in FIG. 9, the motivation index table 900 includes a name of a person, “frequency” that is an index of the number of times selected as a target, “recovery” that is an index of recovery of a time index and an emotional index, and a motivation index. “Index” indicating the breakdown, the value of the index, and “value / MAX” which is the maximum value are included as information items. In the motivation index table 900, the motivation index is omitted and expressed as “mochi index”. In addition, the maximum value is represented by the sum of an elapsed time index (expressed as “time index”) having a maximum value of “50” and an action index (expressed as “emotional index”) having a maximum value of “50”. A motivation index of “100” was defined. The motivation index table 900 displays values such as the motivation index as a bar graph.

[Configuration of approach behavior control means]
As shown in FIG. 10, the approach behavior control unit 48 includes an environment information detection unit 110, a human situation determination unit 120, and a response behavior control unit 130, by which various types of information stored in the storage unit 30 and The control described later is performed based on the data.

  The local map data storage means 32 of the storage unit 30 includes, in addition to the local map (see FIG. 3) described above, utterance output information and map indicating at least one of the availability of the utterance of the robot R, the utterance volume, and the utterance tone. Map data created in advance in association with position information indicating the upper position is stored. In the present embodiment, the map data is created by associating speech output information with position information for each preset noise level.

In addition, the storage unit 30 stores image conditions indicating a person's situation, voice conditions indicating a result of voice recognition, and person situation data created in advance by associating speech output information.
Here, the situation of a person refers to a situation in which the person is concentrated on something that should be done now, or various situations in which a person feels embarrassed, annoyed, or uncomfortable when spoken by another person. Further, the situation of a person differs for each person, and even for the same person, it varies depending on various conditions such as time, place, and scene. Therefore, in this embodiment, as an example of a universal situation in which everyone feels that they do not want to be spoken by others as the situation of a person, “the person is resting (sleeping)” It was configured to correspond to two cases of “a person is already in conversation”.

The person status data indicating that the person is resting includes, for example, a resting image condition characterizing that the person is resting, a voice condition indicating that the voice of the person is not detected, and the robot R's speech failure. This is data configured in association with information indicating permission.
The resting image condition is that, for example, in a captured image, when a person has closed eyes for a predetermined time, a state in which the person is facing down continues for a predetermined time compared to when the person is facing the front. The position of the face of the person is lower than the position of the face when the person is sitting for a predetermined time.

The person status data indicating that a person is in conversation includes, for example, an in-conversation image condition that characterizes that a plurality of persons are in conversation, a voice condition that indicates that no person's voice is detected, and an utterance of the robot R. This is data configured in association with information indicating that the information is not permitted.
The in-dialog image condition refers to, for example, a case where a person is opening and closing a mouth in a captured image, and a case where the face direction and line-of-sight direction of two persons are opposite directions.

  Here, in the human situation data, the conditions associated with the information indicating that the speech is not permitted are not limited only to the image condition and the audio condition. As conditions, for example, location, time, ambient sound volume (noise level), person reaction, or a combination thereof can be associated. In this case, the person status data indicating that the person is resting indicates, for example, each condition of the place, time, noise level, reaction of the person when the robot R calls, and disapproval of the robot R's utterance. Information may be associated with the information. Specifically, when “place where the person is = rest place”, AND, “time = lunch break”, AND, “noise level = Low”, AND, “reaction = 0”, “the person is resting” It can be determined. Note that utterance disapproval does not disallow utterances from the robot (say), but does not permit talking (talk). Then, when the other condition indicates “the person is resting”, the robot R finally calls the name of the person.

Next, the configuration of the approach behavior control means 48 will be described in detail.
<Environmental information detection means>
As shown in FIG. 10, the environment information detection unit 110 obtains the utterance output information corresponding to the current position detected by, for example, the GPS receiver SR2, from the map data stored in the local map data storage unit 32, and the environment of the robot R. It is detected as information resulting from. The detected utterance output information is output to the response action control means 130.

<Person status determination means>
The human situation determination unit 120 includes image conditions used to create the personal situation data stored in the storage unit 30, the processing result of the captured image processed by the image processing unit 10, and the voice recognition unit 21b. Based on the recognized result, the situation of the target person is determined. The human situation determination unit 120 includes an individual situation estimation unit 121, a group situation estimation unit 122, an utterance timing determination unit 123, and an interest estimation unit 124.

The personal situation estimation unit 121 determines whether or not the target person is resting based on the resting image condition, the processing result of the captured image, and the result of voice recognition. Here, the processing result of the photographed image includes the position of the face recognized by the face recognition unit 11c of the image processing unit 10, and includes information on whether the eyes determined by the line-of-sight detection unit 11d are closed. . The result of voice recognition includes information about whether or not a person's voice has been input.
The group situation estimation unit 122 determines whether or not another person is recognized in the vicinity of the target person, and when the other person is recognized, based on the image condition during conversation and the processing result of the captured image. Thus, it is determined whether or not a plurality of persons including the target person are in conversation. As a method for recognizing a plurality of persons, any one of identification by the object data integration unit 42, identification by image processing by the image processing unit 10, and identification by tag detection by the target detection unit 80 (see FIG. 6) is used. May be.
The utterance timing determination means 123 is a case where it is determined that the target person is not resting and is not in conversation, and the line-of-sight direction detected by the line-of-sight detection unit 11d is directed toward the robot R. It is determined that it is utterance timing. Here, the case where the line-of-sight direction is facing the direction of the robot R does not only indicate that the line-of-sight direction is completely coincident with the direction of the robot R, but the line-of-sight direction is within a preset range. It only has to be.

The interest estimation unit 124 calculates an interest level obtained by quantifying the interest expressed by the person with respect to the topic uttered after uttering the target person with respect to a predetermined topic, and the calculated interest level increases. When the degree of interest increases, it is determined that the person is interested in the topic, and the determination result is recorded in the storage means 30.
In the present embodiment, the interest estimation unit 124 calculates, as a method of calculating the degree of interest, an image determination mode based on an image of a person, a sound determination mode based on an input person's voice, and an integrated determination that integrates them. Modes, and each mode is configured to be appropriately switched.

Specifically, in the image determination mode, the interest estimation unit 124 expresses the interest expressed by a vector indicating the line-of-sight direction detected by the line-of-sight detection unit 11d after uttering an intended person with respect to a predetermined topic. Calculate the degree. In this case, the interest estimation unit 124 calculates the degree of interest so that the maximum value is obtained when the detected gaze direction matches the direction of the robot R.
Further, in the voice determination mode, the interest estimation unit 124 recognizes a keyword or a phrase recognized by the voice recognition unit 21b after speaking about a predetermined topic to the target person, or a speech information detecting hand. The degree of interest is calculated by quantifying the person utterance information of the volume, sound quality, and speech speed detected by the unit 21e. In this case, the interest estimation means 124 calculates the degree of interest so that the greater the number of detected keywords, the greater the volume, the higher the sound quality, and the faster the speech speed, the greater the interest. For example, a predetermined weight may be assigned to each keyword such as “interesting” or a phrase such as “tell me in detail”.
Further, in the integrated determination mode, the interest estimation unit 124 calculates the final interest by adding a predetermined weight to the interest degree calculated in the image determination mode and the interest degree calculated in the voice determination mode. Calculate as degrees.

<Responding behavior control means>
The response action control means 130 extracts the utterance output information corresponding to the situation of the person determined by the human situation determination means 120 and the reaction from the person from the person situation data stored in the storage unit 30, and the extracted utterance output information And utterance output information detected by the environment information detection means 110, at least one of utterance availability, utterance volume, and utterance tone for the target person is determined. The response action control means 130 includes an utterance level adjustment means 131, a gesture adjustment means 132, a topic control means 133, and an action general control means 134.

  The utterance level adjusting unit 131 adjusts the level of the utterance volume or switches the utterance tone based on the integrated value calculated by the behavior control unit 134 when the utterance to the person is permitted by the behavior control unit 134. Is what you do. Here, the adjustment of the level of the utterance volume is, for example, an instruction to reduce the initial volume level specified by the utterance action command output to the speech synthesizer 21a (see FIG. 6) by 50% or 50%, for example. It is to output an instruction to raise. In the present embodiment, the utterance level adjusting unit 131 adjusts the level of the utterance volume in consideration of the distance from the position where the robot R actually speaks to the position where the target person is located when the utterance is permitted. adjust. That is, the utterance level adjusting means 131 increases the utterance volume level when the person is farther than the preset utterance distance range, and the utterance volume level when the person is closer to the utterance distance range. Make it smaller. The distance to the position where the target person is present is detected by, for example, the stereo processing unit 11a (see FIG. 6), detected by the target detection unit 80 (see FIG. 6), or by an object map (see FIG. 8). ). The switching of the utterance tone is to switch the initial tone specified by the utterance action command output to the speech synthesizer 21a (see FIG. 6) to another tone. In the present embodiment, normal tone data and special tone data stored in the storage unit 30 are appropriately switched.

  The gesture adjusting unit 132, when the utterance volume level is adjusted by the utterance level adjusting unit 131, is proportional to the adjusted utterance volume level, and at least of the head R1, the arm R2, and the leg R3 by gesture. The movement width of any part is adjusted. This gesture adjustment means 132 adjusts the movement width of each part described in the command output for designating gestures from the gesture integration unit 44 to the autonomous movement control unit 50. For example, when the utterance volume level is reduced by 50%, the initial movement range described in the command is similarly reduced by 50%, and conversely, when the utterance volume level is increased by 50%, Similarly, the initial movement width described in (1) is increased by 50%. The movement includes linear movement and rotational movement.

  The topic control unit 133 provides the topic stored in the person information storage unit 35, and when the interest estimation unit 124 determines that the subject person is not interested in the provided topic (degree of interest) Utterances about the topic being provided). In the present embodiment, the topic control unit 133 switches the topic stored in the person information storage unit 35 based on the degree of interest calculated by the interest estimation unit 124, and switches the topic for the target person. Talk about.

The behavior overall control unit 134 determines whether or not the utterance is permitted by integrating the utterance output information extracted from the person situation data and the utterance output information detected by the environment information detection unit 110. When it is determined to be permitted, an approach behavior indicating an integrated speech volume or speech tone is determined. In this embodiment, the behavior control unit 134 prioritizes the utterance output information detected by the environment information detection unit 110 over the utterance output information extracted from the person situation data when determining the approach behavior. That is, the behavior control unit 134 first attaches importance to the environment around the robot R, extracts the utterance availability, the utterance volume, and the utterance tone from the utterance information detected by the environment information detection unit 110, and can speak. It is determined whether or not there is. Next, the behavior overall control means 134 determines whether or not the speech can be made by adding the situation of the person recognized on the spot to the determination material. Then, the behavior control unit 134 sets the utterance volume and utterance information detected by the environment information detection unit 110 as final integrated values. Further, the behavior overall control unit 134 determines whether or not any of the following four items is satisfied, and ends the approach behavior for the target person when the termination condition is satisfied.
First ending condition: When an environment capable of speaking cannot be established even after a preset time has elapsed.
Second end condition: When the target person is still in conversation even after a preset time has elapsed.
Third end condition: When the target person is resting even after a preset time has elapsed.
Fourth end condition: When the end timing is set in advance. Note that the fourth end condition includes a case where battery replenishment is required or a time for executing a task is reached.

[Robot motion]
The operation of the robot R shown in FIG. 6 will be described with reference to FIG. 11 (refer to FIGS. 1, 6, 7, and 10 as appropriate), mainly focusing on the operation of the approach behavior control means 48. FIG. 11 is a flowchart showing the operation of the robot R shown in FIG. The robot R previously acquires information such as a local map from the management computer 3 via the wireless communication unit 60 by the main control unit 40. In this embodiment, the robot R moves to the location of the person selected by the action plan added by the motivation management unit 47 of the main control unit 40 when it does not have a task to be executed at present. It is assumed that the approach behavior control means 48 of the main control unit 40 starts operating before speaking to interact with the person. Further, it is assumed that the interest estimation unit 124 of the approach behavior control unit 48 is set to an image determination mode based on an image of a person as a method of calculating the degree of interest.

  Then, the approach behavior control means 48 causes the utterance output information corresponding to the current position and noise level detected from the map data stored in the local map data storage means 32 by the environment information detection means 110 to originate from the environment. It detects as information (step S1). Then, the approach behavior control unit 48 determines whether or not the current position is an utterable environment based on the utterance output information corresponding to the current position and the noise level by the behavior control unit 134 of the response behavior control unit 130. (Step S2).

  In step S2, when the current position is an utterable environment (step S2: Yes), the approach behavior control unit 48 uses the group situation estimation unit 122 of the human situation determination unit 120 to place another target person in the vicinity of the target person. It is determined whether or not a person is recognized. That is, the group status estimation unit 122 determines whether or not the target person is alone (one person) (step S3). When the target person is not alone (one person) (step S3: No), subsequently, the group situation estimation unit 122 determines whether or not a plurality of persons including the target person are in conversation (step S3). Step S4).

  In step S4, when a plurality of persons including the target person are not in conversation (step S4: No), the personal situation determination unit 120 uses the personal situation estimation unit 121 to check whether the target person is resting. It is determined whether or not (step S5). In step S3, when the target person is alone (one person) (step S3: Yes), the human situation determination unit 120 skips step S4 and proceeds to step S5.

  In step S5, when the target person is not resting (step S5: No), the person situation determination unit 120 determines the utterance timing by the utterance timing determination unit 123 (step S6). Note that the utterance timing determination unit 123 determines the utterance timing after confirming that the gaze direction of the person faces the direction of the robot R. The approach behavior control unit 48 integrates the utterance output information detected in step S1 by the utterance level adjustment unit 131 when the behavior control unit 134 of the response behavior control unit 130 determines that the utterance is permitted. When the utterance volume level is adjusted or the utterance tone is switched as a value, and the utterance is accompanied by gestures and the utterance volume level is adjusted, the gesture adjustment means 132 causes the utterance volume level to be proportional to the utterance volume level. The movement width of each part by gesture is adjusted (step S7).

  Then, the robot R actually performs the utterance. That is, the topic control means 133 of the response action control means 130 instructs the speech synthesizer 21a to output speech based on the scenario stored in the scenario storage means 34, and integrates the gesture execution associated with the utterance. The unit 44 is instructed. As a result, the robot R utters and performs gestures associated with the utterance (step S8). Specifically, for example, when the robot R speaks to a target person in the conversation room 307 shown in FIG. 3, the robot R speaks at a relatively high volume in a colloquial tone, and the arm R2 Move relatively large. Further, for example, in the conference room 308 shown in FIG. 3, when speaking to a target person, speaking in a respectful tone and a relatively low volume, the arm portion R2 and the like are moved relatively small. Then, the robot R recognizes the input voice of the person by the voice recognition unit 21b (step S9). Then, the approach behavior control unit 48 calculates the degree of interest based on the detected gaze direction by the interest estimation unit 124 of the human situation determination unit 120, and estimates whether the target person is interested in the topic. (Step S10).

  Then, the approach behavior control unit 48 determines whether or not the termination condition is satisfied by the behavior control unit 134 of the response behavior control unit 130 (step S11). When the end condition is satisfied (step S11: Yes), the approach behavior control unit 48 ends the process. On the other hand, when the end condition is not satisfied (step S11: No), the approach behavior control means 48 is based on the degree of interest estimated by the interest estimation means 124 by the topic control means 133 of the reception behavior control means 130. Next, the topic to be provided is developed (step S12), and the process returns to step S1.

  In the above-described step S2, when the current position is not an utterable environment (step S2: No), the approach behavior control unit 48 proceeds to step S11 and determines whether or not an end condition is satisfied. Further, in the above-described step S4, when a plurality of persons including the target person are in conversation (step S4: Yes), the approach behavior control unit 48 proceeds to step S11. In step S5 described above, when the target person is resting (step S5: Yes), the approach behavior control unit 48 proceeds to step S11.

[Specific examples of topic development]
Here, a specific example of topic development will be described with reference to FIG. 12 (see FIG. 10 as appropriate). For example, nine types of topics are stored in the person information storage unit 35. These topics can be described using, for example, nine nodes 1201 to 1209 and links connecting them as shown in FIG. A node 1201 indicates a topic related to “sports”. The node 1201 is connected to the nodes 1202 to 1204, respectively. Nodes 1202 to 1204 indicate topics related to “land”, “swimming”, and “ball game”, respectively. The robot R starts providing a topic from the node 1201, for example, by the topic control unit 133, and when the topic is provided at the node 1201, for example, according to the average value of the degree of interest during the topic provision. A topic to be provided next is selected from 1202 to 1204. The other nodes 1205 to 1209 are the same as the nodes 1201 to 1204 in FIG. According to this, since the robot R utters information presumed to be preferred by the person without using many questions, the conversation becomes natural and the person of the conversation partner can easily feel familiar with the robot R.

  As another example, each of the nodes 1201 to 1209 may include a question asking whether or not the node's theme is liked and information on the theme of the node. In this case, for example, when the topic control unit 133 starts providing a topic from the node 1201, the robot R asks the question “Do you like sports?” As an initial topic, for example, at the node 1201. Select and speak. Then, when the robot R recognizes the reply “Yes”, for example, the node 1204 selects the question “Do you like ball games?” From the nodes 1202 to 1204 connected to the node 1201. And speak. Similarly, the user selects and speaks the questions “Do you like baseball?” And “Do you like Giants?” As described above, when the answer “Yes” is recognized by voice to all the questions, the topic control unit 133 finally determines the theme as “Giants”, and subsequently, the topic of information on “Giants”. Will be spoken. In the middle, when the answer of “No” is voice-recognized, the topic control means 133 makes a similar question about another theme at the same level as the questioned question. For example, when the robot R recognizes a reply of “No” to the question “Do you like ball games?”, The corresponding question is selected from the node 1202 or the node 1203. According to this, since a person's interest is narrowed down by repeating a question, the information which a person likes can be estimated in a short time, and information can be provided efficiently.

  As another example, a result obtained by investigating a predetermined person's interest in advance through a questionnaire, for example, can be associated with each node 1201 to 1209 for each person. In this case, the robot R can utter a topic in a different manner of development for each recognized person, based on the results of a prior investigation. Furthermore, as another example, the topic control unit 133 may randomly select any one of the nodes 1201 to 1209.

  According to the present embodiment, the robot R captures the current location of the current position and the measured noise and the current image of the target person in order to control the approach behavior to the target person before speaking. Depending on the situation indicated by the image and the speech recognition result, it is possible to change the availability of speech, speech volume, and speech tone. Further, since the robot R adjusts the movement width of each part by gesture in proportion to the adjusted level of the utterance volume, the gesture during utterance can be made natural. Further, the robot R determines whether the person who is talking to the topic is interested based on the degree of interest calculated after the utterance about the topic, and estimates the topic that the person likes and utters the topic. be able to.

  As mentioned above, although preferable embodiment of this invention was described, this invention is not limited to above-described embodiment. For example, in this embodiment, the behavior control unit 134 prioritizes the utterance output information detected by the environment information detection unit 110 over the utterance output information extracted from the person situation data when determining the approach behavior. However, the utterance output information extracted from the person situation data may be prioritized. In this case, the behavior control unit 134 first places importance on the status of the recognized person, and then adds the environment around the robot R to the determination material.

  Further, the behavior control unit 134 may be configured to integrate the utterance output information detected by the environment information detection unit 110 and the utterance output information extracted from the person situation data at the same timing. In this case, for example, the behavior control unit 134 compares the utterance output information extracted from the person situation data with the utterance output information detected by the environment information detection unit 110. The integrated value obtained by digitizing the output information and performing weighting to calculate an integrated value is calculated. When the calculated integrated value is smaller than a preset setting value indicating permission of utterance, utterance to a person is permitted. In addition, when the utterance output information extracted from the person situation data and the utterance output information detected by the environment information detection unit 110 are the same, the behavior overall control unit 134 determines whether the robot R is based on the utterance output information. The speech synthesis unit 21a is instructed of the speech volume and speech tone based on the speech output information when it is determined that speech is possible.

  Further, in the present embodiment, the map data stored in the local map data storage means 32 is assumed that the utterance output information is created in association with the position information for each preset noise level. It is not limited to. For example, the map data may be created by associating the utterance output information with the position information for each information such as a time zone or predetermined event information.

  The map data does not necessarily have to be created for each noise level, and may be created by associating the utterance output information and the position information. In this case, a table in which the noise level and the utterance output information are associated with each other is created separately, and the environment information detecting unit 110 detects the noise level detected by the noise measuring unit 21d of the voice processing unit 20 and the map data. The utterance output information corresponding to the current position can be determined according to a predetermined rule with reference to a table created separately and output to the response action control means 130.

  In this embodiment, in the image determination mode, the degree of interest is calculated from the line-of-sight direction. However, the degree of interest may be calculated using the face direction and the distance from the robot R to the person's face. Good. Here, the distance from the robot R to the person's face indicates how the person drawn into the topic gets on. Also, the degree of interest may be calculated by digitizing the facial expression and whether or not it is whispering.

  In the present embodiment, the robot has been described as an autonomous mobile robot capable of walking on two legs. However, the present invention is not limited to this, and various mobile objects such as an autonomous mobile robot that moves by wheels, an industrial robot, and an automobile. Application to is also possible.

It is a figure which shows typically the structure of the robot system containing the robot which concerns on embodiment of this invention. It is a figure which shows typically an example of the self position detection and object detection by a robot. It is a figure which shows the example of the local map used with the robot system shown in FIG. It is a figure which shows an example of the task information database memorize | stored in the memory | storage means of the management computer shown in FIG. It is a figure which shows an example of the task schedule table memorize | stored in the memory | storage means of the management computer shown in FIG. It is a block diagram which shows the structure of the robot which concerns on embodiment of this invention. It is a block diagram which shows the structure of the main control part of the robot shown in FIG. It is a figure which shows an example of object data. It is a figure which shows an example of the motivation index data. It is a block diagram which shows the structure of the approach action control means shown in FIG. It is a flowchart which shows operation | movement of the robot shown in FIG. It is a figure which shows typically the example of expansion | deployment of a topic.

Explanation of symbols

A Robot system R Robot R1 Head R2 Arm R3 Leg R4 Torso R5 Back storage 1 Base station 2 Robot dedicated network 3 Management computer 3a Storage unit 4 Network 5 Terminal 10 Image processing unit (image processing means)
11a Stereo processing unit 11b Moving body extraction unit 11c Face recognition unit (face recognition means)
11d Line-of-sight detection unit (line-of-sight detection means)
20 Voice processing unit (voice processing means)
21a Speech synthesis unit 21b Speech recognition unit (speech recognition means)
21c Sound source localization unit 21d Noise measurement unit (noise measurement means)
21e Utterance information detection unit (utterance information detection means)
30 storage unit (storage means)
31 Object data storage means 32 Local map data storage means 33 Motivation index storage means 34 Scenario storage means 35 Person information storage means 40 Main control part 41 Stationary obstacle integration part 42 Object data integration part 43 Behavior pattern part 44 Gesture integration part (gesture Integration method)
45 Internal state detection unit 46 Action plan management unit 47 Motivation management unit 48 Approach action control means 50 Autonomous movement control part (autonomous movement control means)
60 wireless communication unit 70 battery 80 target detection unit (target detection means)
DESCRIPTION OF SYMBOLS 90 Peripheral state detection part 110 Environmental information detection means 120 Person situation judgment means 121 Individual situation estimation means 122 Group situation estimation means 123 Utterance timing determination means 124 Interest estimation means 130 Response action control means 131 Utterance level adjustment means 132 Gesture adjustment means 133 Topic Control means 134 Action control means C Camera (photographing means)
MC Microphone S Speaker (Audio output means)
SR1 Gyro sensor SR2 GPS receiver (self-position detection means)

Claims (7)

Image processing for detecting the current position of the robot on a preset map and image processing so that the situation of the person can be discriminated from a photographed image obtained by photographing the person to be communicated with the photographing means Means, speech processing means for performing speech recognition and speech so that the situation of the person can be discriminated from speech, and approach behavior control means for controlling approach behavior to the target person before performing the speech. Robot,
Map data created by associating utterance output information indicating at least one of utterance availability, utterance volume and utterance tone set in advance with position information indicating the position on the map;
Storage means for storing the utterance output information and human condition data created by associating a preset image condition indicating a person's condition and a sound condition indicating a reaction from the person,
The approach behavior control means includes:
Environmental information detecting means for detecting speech output information corresponding to the detected current position as information resulting from the environment of the robot from the map data;
Based on the preset image condition, the processing result of the captured image, and the result of the voice recognition, a human situation determination unit that determines the situation of the target person;
The utterance output information corresponding to the situation of the person determined by the person situation determination means is extracted from the person situation data, and based on the extracted utterance output information and the utterance output information detected by the environment information detection means And a response behavior control means for determining at least one of speech availability, speech volume and speech tone for the subject person. Noise measuring means for measuring the noise around the robot and detecting the noise level;
The map data is created by associating the utterance output information with the position information for each preset noise level,
The environment information detection means is information derived from the environment of the robot from the map data created for each preset noise level, with utterance output information corresponding to the detected current position and the detected noise level. The robot according to claim 1, wherein the robot is detected as: The response action control means includes:
Comparing the utterance output information extracted from the person situation data with the utterance output information detected by the environment information detection means, and when the two are different, the utterance output information is digitized and weighted and integrated. Calculating a value, and when the calculated integrated value is smaller than a preset value indicating permission of utterance set in advance, behavior overall control means for allowing utterance to the person,
An utterance level adjusting means for adjusting an utterance volume level or switching an utterance tone based on the integrated value when utterance to the person is permitted;
The robot according to claim 1, further comprising: Autonomous movement for autonomously moving the at least one part by outputting a drive signal to a driving means for driving at least one part of the head, arm and leg respectively connected to the torso of the robot Control means;
Scenario storage means for storing a pre-created scenario for designating a gesture that is a body motion that moves the at least one part when performing a predetermined utterance;
Gesture integration means for extracting gestures corresponding to utterances made to the target person from the scenario, and outputting a command for specifying the extracted gestures to the autonomous movement control means;
Gesture adjusting means for adjusting the movement width of the part by the gesture specified as the command in proportion to the adjusted utterance volume level when the utterance volume level is adjusted by the utterance level adjusting means;
The robot according to claim 3, further comprising: The image processing means includes
Face recognition means for extracting a face area of the target person from the captured image;
Gaze detection means for detecting the gaze direction of the target person from the extracted face area;
After the utterance on the predetermined topic is started for the target person, the degree of interest is calculated by quantifying the gaze direction detected by the gaze detection means, and it is determined whether or not the calculated degree of interest has increased. And when the degree of interest rises, it is determined that the person is interested in the topic, and interest estimation means for recording the determination result;
Topic control means for interrupting utterances related to the predetermined topic when the degree of interest decreases,
The robot according to any one of claims 1 to 4, further comprising: The voice processing means is
Speech information detecting means for detecting person speech information indicating at least one of the volume, sound quality, and speech speed of the target person's voice from the input voice;
After the utterance on a predetermined topic is started for the target person, a recognition result obtained by voice recognition by the voice processing unit or a person's utterance information detected by the utterance information detection unit is calculated as a degree of interest. The interest estimation means for determining whether or not the calculated interest level has increased, and determining that the person is interested in the topic when the interest level has increased, and recording the determination result When,
Topic control means for interrupting utterances related to the predetermined topic when the degree of interest decreases,
The robot according to any one of claims 1 to 4, further comprising: A personal information storage means for storing a plurality of topics;
6. The topic control unit switches a topic stored in the person information storage unit based on the degree of interest, and performs an utterance on the switched topic for the target person. Alternatively, the robot according to claim 6.

Images