JP2005254433A - Personal robot and personal robot control program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To more smoothly have communication between a user and a personal robot. <P>SOLUTION: The personal robot communicating with a human, has an image pickup part picking up the image of an object and an opening part which is installed at the outside of the image pickup part and picks up the image by the image pickup part. The personal robot is also equipped with a shell part capable of changing the position of the opening part, and an image pickup range control part which controls the range picked up with the image pickup part by controlling the position of the opening part of the shell part. The shell part intercepts a partial light of the incident light into the image pickup part and is installed as at least a part of eyeballs of the personal robot. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to an interpersonal robot and an interpersonal robot control program. In particular, the present invention relates to an interpersonal robot that communicates with a person and an interpersonal robot control program that controls the interpersonal robot.

In recent years, various robots have been developed with the development of robot engineering technology. Among them, interpersonal robots that are used for applications such as pets and customer service and communicate with humans have become a hot topic in mass media and the like.
Some conventional interpersonal robots have a function of recognizing a general direction in which a voice is heard when talking to a user and communicating with the user with the body directed in that direction. .

  Since the existence of the prior art document is not recognized at the present time, the description regarding the prior art document is omitted.

  However, in a conventional interpersonal robot, when a large number of users are talking at the same time, if the interpersonal robot changes the direction of the body each time the target to be communicated is switched, a certain amount of time is required for the switching operation. As a result, there was a problem that communication was not performed smoothly. Also, if the interpersonal robot does not change the body direction even if the target to be communicated is changed and is facing a certain direction, the user determines with which user the interpersonal robot is communicating. There was a problem that it was difficult.

  Accordingly, an object of the present invention is to provide an interpersonal robot and an interpersonal robot control program that can solve the above-described problems. This object is achieved by a combination of features described in the independent claims. The dependent claims define further advantageous specific examples of the present invention.

  In order to solve the above-described problem, in the first embodiment of the present invention, an interpersonal robot that communicates with a human being, provided with an imaging unit that captures an image of a subject, an outside of the imaging unit, and an imaging unit Has an opening for capturing an image, and a shell part that can change the position of the opening part, and a position of the opening part of the shell part is controlled to control a range in which the imaging part captures an image. An imaging range control unit. The shell portion may block part of the light incident on the imaging unit. The shell part may be provided as at least a part of the eyeball of the interpersonal robot. The interpersonal robot further includes a speaker selection unit that selects an area including a person talking to the interpersonal robot in an image captured by the imaging unit, and the imaging range control unit is configured by the speaker selection unit. Based on the selected range, the range captured by the imaging unit may be controlled.

  Further, in the second embodiment of the present invention, there is provided an interpersonal robot control program for causing an interpersonal robot to function, wherein the interpersonal robot is provided on an outside of the image pickup unit, an image pickup unit for picking up an image of a subject, and an image pickup unit Has an opening for capturing an image, and a shell part that can change the position of the opening part, and a position of the opening part of the shell part is controlled to control a range in which the imaging part captures an image. It functions as an interpersonal robot including an imaging range control unit. The interpersonal robot further includes a speaker selection unit that selects an area including a person talking to the interpersonal robot in an image captured by the imaging unit, and the imaging range control unit is configured by the speaker selection unit. Based on the selected range, the range captured by the imaging unit may be controlled.

  The above summary of the invention does not enumerate all the necessary features of the present invention, and sub-combinations of these feature groups can also be the invention.

  ADVANTAGE OF THE INVENTION According to this invention, a user and an interpersonal robot can communicate more smoothly.

  Hereinafter, the present invention will be described through embodiments of the invention. However, the following embodiments do not limit the invention according to the scope of claims, and all combinations of features described in the embodiments are included. It is not necessarily essential for the solution of the invention.

  FIG. 1 shows an outline of an interpersonal robot 10 according to an embodiment of the present invention. The interpersonal robot 10 performs two-way communication such as a response using voice, characters, images, or the like when spoken by a user. Specifically, the interpersonal robot 10 recognizes the user's movement and facial expression based on an image input using a digital camera or the like, and recognizes the user's spoken voice based on voice input using a microphone or the like. By doing so, the user's intention is recognized. And the interpersonal robot 10 selects a voice stored in advance based on a result of recognizing the user's intention, or synthesizes a voice as necessary and outputs it using a speaker or the like. Communicate with users autonomously.

  Here, the interpersonal robot 10 uses a display device such as an LCD panel to display characters and images instead of communicating with the user by outputting sound from a speaker. You may communicate. The interpersonal robot 10 transmits the input user's image and speech to the external control terminal instead of autonomously communicating with the user, and receives the operator's voice from the control terminal. You may communicate with a user by outputting using a speaker etc.

  It is preferable that the interpersonal robot 10 has a form imitating a person as shown in the figure or a form imitating an animal such as a dog or a cat because it is more familiar to the user. . The interpersonal robot 10 according to the present embodiment includes an eyeball 15 including a device for inputting a user image, such as a digital camera.

  The interpersonal robot 10 according to the present embodiment controls the eyeball 15 that is an image input device so as to be visible from the outside when communicating with the user, and in which direction the interpersonal robot 10 is looking. That is, it is an object to make communication with a user smoother by informing the user and other users of which user the interpersonal robot 10 is communicating with.

  FIG. 2 is a block diagram showing an example of the configuration of the interpersonal robot 10 according to the embodiment of the present invention. The interpersonal robot 10 includes an imaging unit 100, a primary memory 110, an imaging control unit 120, a sound collection unit 130, a speaker selection unit 140, a shell unit 150, and an imaging range control unit 160. The imaging unit 100 includes an optical system 102, a CCD 104, and an imaging signal processing unit 106, and captures an image of a subject. The optical system 102 includes, for example, a focus lens and a zoom lens, and forms a subject image on the light receiving surface of the CCD 104. The CCD 104 includes a plurality of light receiving elements, and outputs the charges accumulated in the respective light receiving elements by the optical image of the subject formed on the light receiving surface by the optical system 102 to the imaging signal processing unit 106 as an analog voltage signal. . The imaging signal processing unit 106 decomposes the analog voltage signal indicating the subject image received from the CCD 104 into R, G, and B components. Then, the imaging signal processing unit 106 performs A / D conversion on the analog signal decomposed into the R, G, and B components, and digital image data representing the subject image obtained as a result is stored in the primary memory 110. Output. The primary memory 110 is a volatile memory such as a DRAM and stores digital image data output from the imaging signal processing unit 106. The imaging control unit 120 controls the imaging of an image showing a subject by driving a mechanism member included in the imaging unit 100. For example, the imaging control unit 120 controls exposure, white balance, zoom operation, aperture operation, and the like.

  The sound collection unit 130 is, for example, a microphone, inputs voice, and outputs the input voice to the speaker selection unit 140. The speaker selection unit 140 recognizes a person talking to the interpersonal robot 10 in an image indicated by image data stored in the primary memory 110 using an image recognition technique or the like, and an area including the person Select. Here, the speaker selection unit 140 may recognize a person talking to the interpersonal robot 10 based on the voice received from the sound collection unit 130. Then, the speaker selection unit 140 outputs information indicating the selected area to the imaging range control unit 160.

  The shell unit 150 is provided outside the imaging unit 100, has an opening for the imaging unit 100 to capture an image, and can change the position of the opening. The imaging range control unit 160 controls the range that the imaging unit 100 captures by controlling the position of the opening of the shell unit 150. Specifically, the imaging range control unit 160 controls the range that the imaging unit 100 captures based on the range selected by the speaker selection unit 140. For example, the imaging range control unit 160 may control the position of the opening of the shell unit 150 so that the range selected by the speaker selection unit 140 is captured at the center of the image.

  According to the interpersonal robot 10 according to the present embodiment, by controlling the imaging range in the imaging unit 100 using the shell unit 150 provided outside the imaging unit 100, the user can determine which range the interpersonal robot 10 has. It is possible to determine based on the position of the opening in the shell 150 whether the interpersonal robot 10 is communicating with which user. Thereby, the user can perform communication with the interpersonal robot 10 more smoothly.

  In addition, even if the interpersonal robot 10 is talked by being surrounded by a plurality of users and the object to be communicated is frequently switched, the opening of the shell 150 is not changed without changing the orientation of the interpersonal robot 10. By changing the position, the object to be communicated can be switched so as to be discriminable from the user. Thereby, since the interpersonal robot 10 can switch the communication target in a shorter time, the responsiveness can be improved. Furthermore, since the part to be driven when switching the object to be communicated can be reduced, the power consumption of the interpersonal robot 10 can be reduced, and the cost and size of the interpersonal robot 10 can be reduced.

  FIG. 3 shows an example of a cross section of the eyeball 15 according to the embodiment of the present invention. The eyeball 15 includes a shell portion 150, an optical system 102, and a CCD 104. The shell portion 150 is configured to cover the optical system 102 and the CCD 104 with a light-shielding member, for example, and has an opening portion 155 provided by making a hole in a part thereof. And the shell part 150 restrict | limits the imaging range in the imaging part 100 by interrupting | blocking one part light among the light which injects into the optical system 102. FIG. Specifically, the shell unit 150 limits the imaging range in the imaging unit 100 by causing only the light incident through the opening 155 to enter the optical system 102 and blocking the other light. Here, the size of the opening 155 may vary, and may be larger or smaller than the opening 155 shown in the figure.

  The shell unit 150 changes the imaging range in the imaging unit 100 by changing the position of the opening 155 based on the control of the imaging range control unit 160. For example, the shell portion 150 is provided as a bowl-shaped member having a light shielding property with a hole serving as an opening 155 at the bottom, and the optical center in the optical system 102 is driven by an actuator driven by the imaging range control unit 160. It is controlled to be rotatable around. Further, the shell portion 150 may be configured as a member having a light-transmitting part and another part having a light-blocking property, and the light-transmitting part may function as the opening 155. Here, the translucent member used as the opening 155 does not have to be colorless and transparent, and may have some color. Shell part 150 is a member which has translucency as a whole, and may be constituted so that a portion which corresponds to opening 155 may have a different color from other portions.

  Further, the shell portion 150 is configured by a liquid crystal panel using a light-transmitting electrode, and transmits light at a portion corresponding to the opening 155 and controls the electrode to block light at portions other than the opening 155. Thus, the imaging range in the imaging unit 100 may be controlled.

  FIG. 4 shows an example of a change in position of the opening 155 according to the embodiment of the present invention. FIG. 4A shows an example of the external appearance of the interpersonal robot 10 when the opening 155 is positioned on the left as viewed from the user. In this case, the imaging unit 100 captures an image of the subject on the right side when viewed from the interpersonal robot 10. FIG. 4B shows an example of the appearance of the interpersonal robot 10 when the opening 155 is located on the right side when viewed from the user. In this case, the imaging unit 100 captures an image of a subject on the left side when viewed from the interpersonal robot 10.

  As is clear from FIGS. 4A and 4B, the user can easily determine in which direction the interpersonal robot 10 is capturing an image based on the position of the opening 155. Can do. Furthermore, since the shell unit 150 is provided as at least a part of the eyeball 15, the user can determine which direction the interpersonal robot 10 is imaging in the same sense as the determination of the gaze direction in humans. It can be easily judged. The user can determine whether or not the interpersonal robot 10 is communicating with itself by determining whether or not the interpersonal robot 10 is looking at itself. That is, the user can communicate with the interpersonal robot 10 through eye contact. Thereby, the user can perform communication with the interpersonal robot 10 more smoothly.

  Further, when a wide-angle lens such as a fish-eye lens is used in the optical system 102, the position of the opening 155 is blocked by using the shell 150 to block light that is incident from a direction different from the required direction. It is possible to prevent so-called stray light and to capture a clearer image while ensuring a wide range that can be imaged by changing.

  FIG. 5 is a flowchart showing an example of a process flow in the interpersonal robot 10 according to the embodiment of the present invention. First, the sound collection unit 130 inputs sound (S1000). Here, the sound collection unit 130 may be configured by a plurality of microphones, and may input sound at different positions using the respective microphones. Subsequently, the speaker selection unit 140 determines whether or not the voice input by the sound collection unit 130 includes the user's speaking voice (S1010). For example, the speaker selection unit 140 decomposes a signal indicating the input speech into a plurality of frequency components using a known technique such as fast Fourier transform, and the level of the component in a frequency band specific to human voice is reduced. When it is equal to or greater than a predetermined reference value, it is determined that the input voice includes spoken voice. In addition, when the speaker selection unit 140 recognizes a specific word such as a name given to the interpersonal robot 10 from the input voice by using a known voice recognition technique, the speaker selection unit 140 speaks the input voice. May be determined to be included. By doing so, the interpersonal robot 10 can be made to respond only when spoken by a user who wants to communicate with the interpersonal robot 10.

  Here, the speaker selection unit 140 may not only determine whether or not the user's speaking voice is included in the input voice, but may also detect the generation position or propagation direction of the speaking voice. For example, when the sound collection unit 130 is configured by a plurality of microphones, the speaker selection unit 140 determines the speech for the speech input for each microphone, and based on the position where the largest speech is input. The generation position or the propagation direction of the speaking voice may be detected. In addition, when the sound collection unit 130 is configured by at least two microphones, the speaker selection unit 140 uses a known sound localization technique to shift the input timing of sound waves indicating the same spoken voice for each microphone. Based on the above, the generation position of the spoken voice may be detected.

  If the input voice does not include a speaking voice (S1010: No), the interpersonal robot 10 returns the process to S1000 and inputs the voice again. On the other hand, when the input voice includes a speaking voice (S1010: Yes), the imaging range control unit 160 controls the imaging range in the imaging unit 100 (S1020). Here, when the speaking voice generation position is detected by the speaker selection unit 140, the imaging range control unit 160 controls the imaging range in the imaging unit 100 so that the position is included in the imaging range. On the other hand, when the generation position of the speaking voice is not detected, the imaging range control unit 160 determines a predetermined rule such as sequentially moving the imaging range in the right direction, for example, to capture the entire periphery by repeatedly imaging. You may change based on.

  Further, the imaging range control unit 160 may control the imaging range of the imaging unit 100 by changing the orientation of the interpersonal robot 10 as well as changing the position of the opening of the shell unit 150. However, even in such a case, the time required for changing the imaging range can be shortened by changing the position of the opening of the shell 150 as much as possible to control the imaging range. 10 responsiveness is improved.

  Subsequently, the imaging unit 100 captures an image (S1030). Here, in detecting the movement of the mouth, which will be described later, it is preferable that the imaging unit 100 captures a moving image or captures a plurality of temporally continuous still images. Subsequently, the speaker selection unit 140 recognizes whether or not a person is included in the image captured by the imaging unit 100 (S1040). For example, the speaker selection unit 140 recognizes whether or not a person is included based on the distribution of the area indicating the skin color in the captured image. Here, when a moving image is captured by the image capturing unit 100 or when a plurality of still images are captured, the speaker selecting unit 140 performs a recognition process on at least some of the frame images or still images. It's okay. If no person is included in the captured image (S1040: No), the interpersonal robot 10 returns the process to S1000, and causes the sound collection unit 130 to input sound again.

  On the other hand, when a person is included in the captured image (S1040: Yes), the speaker selection unit 140 uses a known image recognition technique such as pattern matching from the image showing the recognized person, for example. The face image is recognized, and the mouth image in the face image is extracted (S1050). Subsequently, the speaker selection unit 140 determines whether or not the mouth image changes in synchronization with the speaking voice (S1060). For example, the speaker selection unit 140 changes the image of the mouth part of a person recognized in a plurality of frame images in a moving image or a plurality of still images that are temporally continuous within a predetermined time interval. If it is, the person is recognized as talking. Then, the speaker selecting unit 140 matches the time zone in which the voice is included in the input voice and the time zone in which the person is speaking at a rate equal to or higher than a predetermined reference value. If it is determined that the mouth image is changing in synchronization with the speaking voice, that is, it is determined that the mouth is moving in synchronization with the speaking voice. Here, when a plurality of persons are included in the captured image, the speaker selection unit 140 detects the mouth image for each of the plurality of persons, and the mouth image is spoken. It may be determined whether or not it has changed in synchronization with the above.

  When the image of the mouth portion does not change in synchronization with the speaking voice (S1060: No), the interpersonal robot 10 returns the process to S1000 and causes the sound collecting unit 130 to input the sound again. On the other hand, when the image of the mouth portion changes in synchronization with the speaking voice (S1060: Yes), the speaker selecting unit 140 recognizes the person corresponding to the image as the person talking to the interpersonal robot 10, An area including at least a part of the person is selected from the captured image (S1070). For example, the speaker selection unit 140 selects a rectangle that circumscribes the person. Then, the imaging range control unit 160 changes the imaging range by controlling the shell unit 150 so that the region selected by the speaker selection unit 140 is arranged at the center of the image captured by the imaging unit 100 (S1080). ).

  As described above, a user who talks to the interpersonal robot 10 by controlling the shell unit 150 to image a region including the user who talks to the interpersonal robot 10 is that the interpersonal robot 10 is looking at himself / herself. Since it can recognize, the communication which made eye contact between a user and the interpersonal robot 10 can be materialized. Thereby, the user can communicate with the interpersonal robot 10 more smoothly.

  In this example, the interpersonal robot 10 detects the user who talks to the interpersonal robot 10 using both the sound input by the sound collecting unit 130 and the image captured by the image capturing unit 100. The user who talked to may be detected using only one of these. For example, the interpersonal robot 10 may detect the speaking user by detecting the generation position of the speaking voice using a plurality of microphones and controlling the imaging range based only on the detected position. Further, for example, the interpersonal robot 10 captures an image by sequentially changing the position of the opening in the shell unit 150 and the direction of the interpersonal robot 10, recognizes the person with respect to the captured image, and also images the mouth portion. The user who talked to may be detected by detecting the change in.

  FIG. 6 is a block diagram illustrating an example of a hardware configuration of a computer 1500 according to the present embodiment. The computer 1500 according to this embodiment is connected to the CPU peripheral unit including the CPU 1505, the RAM 1520, the graphic controller 1575, and the display device 1580 connected to each other by the host controller 1582, and connected to the host controller 1582 by the input / output controller 1584. Input / output unit having communication interface 1530, hard disk drive 1540, and CD-ROM drive 1560, and legacy input / output unit having ROM 1510, flexible disk drive 1550, and input / output chip 1570 connected to input / output controller 1584 With.

  The host controller 1582 connects the RAM 1520 to the CPU 1505 and the graphic controller 1575 that access the RAM 1520 at a high transfer rate. The CPU 1505 operates based on programs stored in the ROM 1510 and the RAM 1520 and controls each unit. The graphic controller 1575 acquires image data generated by the CPU 1505 and the like on a frame buffer provided in the RAM 1520 and displays the image data on the display device 1580. Alternatively, the graphic controller 1575 may include a frame buffer that stores image data generated by the CPU 1505 or the like.

  The input / output controller 1584 connects the host controller 1582 to the hard disk drive 1540, the communication interface 1530, and the CD-ROM drive 1560, which are relatively high-speed input / output devices. The hard disk drive 1540 stores programs and data used by the CPU 1505 in the computer 1500. The communication interface 1530 communicates with the interpersonal robot 10 via the network, and provides programs and data to the interpersonal robot 10. The CD-ROM drive 1560 reads a program or data from the CD-ROM 1595 and provides it to the hard disk drive 1540 and the communication interface 1530 via the RAM 1520.

  The input / output controller 1584 is connected to the ROM 1510, the flexible disk drive 1550, and the relatively low-speed input / output device of the input / output chip 1570. The ROM 1510 stores a boot program executed when the computer 1500 is started up, a program depending on the hardware of the computer 1500, and the like. The flexible disk drive 1550 reads a program or data from the flexible disk 1590 and provides it to the hard disk drive 1540 and the communication interface 1530 via the RAM 1520. The input / output chip 1570 connects various input / output devices via a flexible disk drive 1550 and, for example, a parallel port, a serial port, a keyboard port, a mouse port, and the like.

  A program provided to the communication interface 1530 via the RAM 1520 is stored in a recording medium such as the flexible disk 1590, the CD-ROM 1595, or an IC card and provided by the user. The program is read from the recording medium, provided to the communication interface 1530 via the RAM 1520, and transmitted to the interpersonal robot 10 via the network. The program transmitted to the interpersonal robot 10 is installed and executed in the interpersonal robot 10. The program installed and executed in the interpersonal robot 10 causes the interpersonal robot 10 to function as the interpersonal robot 10 described with reference to FIGS.

  The program shown above may be stored in an external storage medium. As the storage medium, in addition to the flexible disk 1590 and the CD-ROM 1595, an optical recording medium such as a DVD or PD, a magneto-optical recording medium such as an MD, a tape medium, a semiconductor memory such as an IC card, or the like can be used. Further, a storage device such as a hard disk or a RAM provided in a server system connected to a dedicated communication network or the Internet may be used as a recording medium, and the program may be provided to the computer 1500 via the network.

  As mentioned above, although this invention was demonstrated using embodiment, the technical scope of this invention is not limited to the range as described in the said embodiment. It will be apparent to those skilled in the art that various modifications or improvements can be added to the above-described embodiment. It is apparent from the scope of the claims that the embodiments added with such changes or improvements can be included in the technical scope of the present invention.

It is a figure which shows the outline | summary of the interpersonal robot 10 which concerns on embodiment of this invention. It is a block diagram showing an example of composition of interpersonal robot 10 concerning an embodiment of the present invention. It is a figure which shows an example of the cross section of the eyeball 15 which concerns on embodiment of this invention. It is a figure which shows an example of the position change of the opening part 155 which concerns on embodiment of this invention. It is a flowchart which shows an example of the flow of the process in the interpersonal robot 10 which concerns on embodiment of this invention. It is a block diagram which shows an example of a structure of the computer 1500 which concerns on embodiment of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Interpersonal robot, 15 Eyeball, 100 Imaging part, 102 Optical system, 104 CCD, 106 Imaging signal processing part, 110 Primary memory, 120 Imaging control part, 130 Sound collection part, 140 Speaker selection part, 150 Shell part, 155 Aperture, 160 Imaging range controller

Claims (6)

An interpersonal robot that communicates with people,
An imaging unit that captures an image of a subject;
A shell portion that is provided outside the imaging unit, the imaging unit having an opening for capturing the image, and capable of changing a position of the opening;
An interpersonal robot comprising: an imaging range control unit configured to control a range captured by the imaging unit by controlling a position of the opening included in the shell unit.   The interpersonal robot according to claim 1, wherein the shell portion blocks a part of light incident on the imaging unit.   The interpersonal robot according to claim 1, wherein the shell portion is provided as at least a part of an eyeball of the interpersonal robot. A speaker selection unit for selecting a region including a person talking to the interpersonal robot in the image captured by the imaging unit;
The interpersonal robot according to claim 1, wherein the imaging range control unit controls a range captured by the imaging unit based on a range selected by the speaker selection unit. An interpersonal robot control program for functioning an interpersonal robot,
The interpersonal robot
An imaging unit that captures an image of a subject;
A shell portion that is provided outside the imaging unit, the imaging unit having an opening for capturing the image, and capable of changing a position of the opening;
An interpersonal robot control program that functions as an interpersonal robot, comprising: an imaging range control unit that controls a range captured by the imaging unit by controlling the position of the opening of the shell unit. The interpersonal robot further includes a speaker selection unit that selects an area in the image captured by the image capturing unit that includes a person talking to the interpersonal robot,
The interpersonal robot control program according to claim 5, wherein the imaging range control unit controls a range captured by the imaging unit based on a range selected by the speaker selection unit.

Images