WO2021117441A1 - 情報処理装置、その制御方法及びプログラム - Google Patents

情報処理装置、その制御方法及びプログラム Download PDF

Info

Publication number
WO2021117441A1
WO2021117441A1 PCT/JP2020/042990 JP2020042990W WO2021117441A1 WO 2021117441 A1 WO2021117441 A1 WO 2021117441A1 JP 2020042990 W JP2020042990 W JP 2020042990W WO 2021117441 A1 WO2021117441 A1 WO 2021117441A1
Authority
WO
WIPO (PCT)
Prior art keywords
motion
voice
autonomous mobile
mobile body
module
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2020/042990
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
達馬 櫻井
高橋　正樹
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Sony Group Corp
Original Assignee
Sony Group Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Sony Group Corp filed Critical Sony Group Corp
Priority to US17/781,275 priority Critical patent/US20220410023A1/en
Priority to JP2021563818A priority patent/JPWO2021117441A1/ja
Publication of WO2021117441A1 publication Critical patent/WO2021117441A1/ja
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63H—TOYS, e.g. TOPS, DOLLS, HOOPS OR BUILDING BLOCKS
    • A63H3/00—Dolls
    • A63H3/28—Arrangements of sound-producing means in dolls; Means in dolls for producing sounds
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63H—TOYS, e.g. TOPS, DOLLS, HOOPS OR BUILDING BLOCKS
    • A63H11/00—Self-movable toy figures
    • A63H11/18—Figure toys which perform a realistic walking motion
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L13/00—Speech synthesis; Text to speech systems
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63H—TOYS, e.g. TOPS, DOLLS, HOOPS OR BUILDING BLOCKS
    • A63H2200/00—Computerized interactive toys, e.g. dolls

Definitions

  • This disclosure relates to an information processing device, its control method, and a program.
  • Patent Document 1 discloses a technique of calculating an expected value of the user's attention to output information and controlling information output based on the expected value.
  • the information processing apparatus includes a voice module that outputs sound or voice according to an input action plan, and a motion module that executes an operation according to the input action plan.
  • a body controller for creating an action plan for each of the voice module and the motion module is provided, and the body controller acquires voice data to output voice and motion data to execute an operation, and the voice data and the motion data Based on, the first action plan of the voice module and the second action plan of the motion module are created, the first action plan is input to the voice module, and the second action plan is input to the motion module.
  • the agent device can present various information in response to an inquiry from a user, for example.
  • the above information presentation includes, for example, presentation of recommendation information, schedule, news, etc. to the user.
  • the agent device executes the above operation in response to an instruction command input by the user.
  • Examples of the above-mentioned instruction command include inputting a keyword by voice, pressing a button for executing a function, and the like. Therefore, the information presentation by the agent device as described above is a passive operation, and cannot be said to activate communication with the user.
  • agent devices perform continuous dialogue with the user using voice or the like, but in many cases, they only repeatedly execute passive actions in response to the user's instruction command. It is hard to say that true communication is realized.
  • one of the features of the autonomous mobile body 10 according to the present embodiment is that it actively executes various actions (hereinafter, also referred to as incentive actions) that induce communication with the user.
  • the autonomous mobile body according to the present embodiment can actively present information to the user based on environmental recognition. Further, for example, the autonomous mobile body 10 actively executes various incentive actions that encourage a predetermined action of the user. In this respect, the autonomous mobile body according to the present embodiment is clearly different from the device that performs a passive operation based on an instruction command.
  • the incentive action by the autonomous moving body according to the present embodiment is active and positive interference with the physical space.
  • the autonomous mobile body according to the present embodiment can move in a physical space and perform various physical actions on a user, a living thing, an article, or the like.
  • the user can comprehensively recognize the movement of the autonomous mobile body through visual, auditory, and tactile senses, and simply interacts with the user using voice. It is possible to realize advanced communication compared to the case of performing.
  • the autonomous mobile body 10 according to the present embodiment can be various devices that perform autonomous operations based on environmental recognition.
  • the autonomous mobile body 10 according to the present embodiment is an ellipsoidal agent-type robot device that autonomously travels by wheels will be described as an example.
  • the autonomous mobile body 10 according to the present embodiment realizes various communications including information presentation by performing autonomous operations according to, for example, the user, the surroundings, and its own situation.
  • the autonomous mobile body 10 according to the present embodiment may be a small robot having a size and weight that can be easily lifted by a user with one hand.
  • FIG. 1 is a front view and a rear view of the autonomous mobile body 10 according to the present embodiment.
  • FIG. 2 is a perspective view of the autonomous mobile body 10 according to the present embodiment.
  • FIG. 3 is a side view of the autonomous mobile body 10 according to the present embodiment.
  • 4 and 5 are a top view and a bottom view of the autonomous mobile body 10 according to the present embodiment, respectively.
  • the autonomous mobile body 10 includes two eye portions 510 corresponding to the right eye and the left eye on the upper part of the main body.
  • the eye portion 510 is realized by, for example, an LED or the like, and can express a line of sight, blinking, or the like.
  • the eye portion 510 is not limited to the above example, and may be realized by, for example, two single or independent OLEDs (Organic Light Emitting Diode).
  • the autonomous mobile body 10 includes one or a plurality of cameras 515 above the eye portion 510.
  • the camera 515 has a function of capturing an image of the user and the surrounding environment.
  • the autonomous mobile body 10 may realize SLAM (Simultaneous Localization and Mapping) based on the image captured by the camera 515.
  • SLAM Simultaneous Localization and Mapping
  • the eye portion 510 and the camera 515 according to the present embodiment are arranged on the substrate 505 arranged inside the exterior surface.
  • the exterior surface of the autonomous moving body 10 is basically formed by using an opaque material, but the portion corresponding to the substrate 505 on which the eye portion 510 and the camera 515 are arranged is transparent.
  • a head cover 550 using a translucent material is provided.
  • the autonomous mobile body 10 is provided with a ToF (Time of Flight) sensor 520 at the lower part of the front surface.
  • the ToF sensor 520 has a function of detecting a distance from an object existing in front of the sensor. According to the ToF sensor 520, distances to various objects can be detected with high accuracy, and by detecting a step or the like, it is possible to prevent a fall or a fall.
  • the autonomous mobile body 10 may be provided with a connection terminal 555 of an external device and a power switch 560 on the back surface.
  • the autonomous mobile body 10 can connect to an external device via the connection terminal 555 and perform information communication.
  • the autonomous mobile body 10 according to the present embodiment is provided with two wheels 570 on the bottom surface.
  • the wheels 570 according to this embodiment are driven by different motors 565.
  • the autonomous moving body 10 can realize moving operations such as forward movement, backward movement, turning, and rotation.
  • the wheel 570 according to the present embodiment may be provided so as to be able to be stored inside the main body and protrude to the outside. In that case, the autonomous mobile body 10 can perform a jumping motion by, for example, vigorously projecting the two wheels 570 to the outside.
  • FIG. 5 shows a state in which the wheels 570 are stored inside the main body.
  • FIG. 6 is a schematic view for explaining the internal structure of the autonomous mobile body 10 according to the present embodiment.
  • the autonomous mobile body 10 includes an inertial sensor 525 and a communication device 530 arranged on an electronic substrate.
  • the inertial sensor 525 detects the acceleration and the angular velocity of the autonomous moving body 10.
  • the communication device 530 has a configuration for realizing wireless communication with the outside, and includes, for example, a Bluetooth (registered trademark) and a Wi-Fi (registered trademark) antenna.
  • the autonomous mobile body 10 is provided with a speaker 535 inside the side surface of the main body, for example.
  • the autonomous mobile body 10 can output various sound information including voice by the speaker 535.
  • the autonomous mobile body 10 includes a plurality of microphones 540 inside the upper part of the main body.
  • the microphone 540 collects user utterances and ambient sounds. Further, by providing the autonomous mobile body 10 with a plurality of microphones 540, it is possible to collect sounds generated in the surroundings with high sensitivity and to realize localization of a sound source.
  • the autonomous mobile body 10 includes a plurality of motors 565.
  • the autonomous moving body 10 includes, for example, two motors 565 for driving the substrate on which the eye portion 510 and the camera 515 are arranged in the vertical and horizontal directions, and two motors 565 for driving the left and right wheels 570. And one motor 565 for realizing the forward leaning posture of the autonomous moving body 10 may be provided.
  • the autonomous mobile body 10 according to the present embodiment can express a rich operation by the plurality of motors 565.
  • FIG. 7 is a diagram showing the configuration of the substrate 505 according to the present embodiment.
  • FIG. 8 is a cross-sectional view of the substrate 505 according to the present embodiment.
  • the substrate 505 according to this embodiment is connected to two motors 565.
  • the two motors 565 can drive the substrate 505 on which the eye portion 510 and the camera 515 are arranged in the vertical and horizontal directions.
  • the eye portion 510 of the autonomous moving body 10 can be flexibly moved in the vertical direction and the horizontal direction, and it is possible to express rich eye movements according to the situation and the movement.
  • the eye portion 510 is composed of a central portion 512 corresponding to the iris and a peripheral portion 514 corresponding to the so-called white eye.
  • the central portion 512 expresses an arbitrary color including blue, red, green, and the like, and the peripheral portion 514 expresses white.
  • the autonomous mobile body 10 according to the present embodiment can express a natural eyeball facial expression closer to that of an actual organism by separating the configuration of the eye portion 510 into two.
  • FIGS. 9 and 10 are views showing the peripheral structure of the wheel 570 according to the present embodiment.
  • the two wheels 570 according to the present embodiment are driven by independent motors 565, respectively. According to this configuration, in addition to simple forward and backward movements, it is possible to delicately express movement movements such as turning and rotation on the spot.
  • the wheel 570 according to the present embodiment is provided so as to be able to be stored inside the main body and protrude to the outside. Further, by providing the damper 575 coaxially with the wheel 570 according to the present embodiment, it is possible to effectively reduce the transmission of impact and vibration to the axle and the main body.
  • the wheel 570 according to the present embodiment may be provided with an auxiliary spring 580.
  • the driving of the wheels according to the present embodiment requires the most torque among the driving parts of the autonomous moving body 10, but by providing the auxiliary spring 580, all the driving parts do not use different motors 565. It is possible to standardize the motor 565.
  • FIG. 11 is a diagram for explaining forward leaning of the autonomous mobile body 10 according to the present embodiment.
  • One of the features of the autonomous moving body 10 according to the present embodiment is that it performs moving motions such as back-and-forth motion, turning motion, and rotational motion while maintaining a forward leaning posture.
  • FIG. 11 shows a side view of the autonomous moving body 10 during traveling.
  • one of the features of the autonomous moving body 10 according to the present embodiment is that the autonomous moving body 10 tilts forward by an angle ⁇ in the vertical direction to perform a moving operation.
  • the angle ⁇ can be, for example, 10 °.
  • a heavy component hp is arranged on the back side of the autonomous moving body 10 according to the present embodiment in order to maintain balance in a forward leaning posture.
  • the heavy component hp according to the present embodiment may be a heavier component than other components included in the autonomous mobile body 10, and may be, for example, a motor 565 or a battery. According to the above component arrangement, the posture control is facilitated in a state where the balance is maintained even if the head is tilted forward, the unintentional tipping of the autonomous moving body 10 can be prevented, and stable forward leaning can be realized. ..
  • 13A and 13B are diagrams for explaining the effect of the forward tilting motion of the autonomous mobile body 10 according to the present embodiment.
  • FIG. 13A shows an example of the rotational movement when the autonomous moving body does not take a forward leaning posture.
  • the autonomous moving body 10 when the autonomous moving body 10 does not take a forward leaning posture and performs a moving motion such as rotation or back-and-forth movement while keeping the oblong body upright, the oblong body does not feel directional. , It is difficult to wipe out the impression that the autonomous moving body is an artificial object.
  • the autonomous mobile body 10 is characterized in that it performs a moving motion such as rotation while maintaining a forward leaning posture.
  • the front upper part of the autonomous moving body 10 is pronounced of the head, and the rear lower part is pronounced of the waist, so that even a simple oblong body is oriented.
  • the structure corresponding to the body part possessed by a human can be expressed by a relatively simple exterior, and the simple form can be anthropomorphized. Therefore, it is possible to give the user an impression as a living organism that goes beyond a mere artificial object.
  • the forward tilting motion according to the present embodiment makes it possible to richly express the facial expression of a robot having a relatively simple exterior such as an ellipsoid, and is complicated like an actual living thing. It can be said that it is a very effective means that can remind us of the movement.
  • the configuration example of the autonomous mobile body 10 according to the first embodiment of the present disclosure has been described in detail above.
  • the above configuration described with reference to FIGS. 1 to 13B is merely an example, and the configuration of the autonomous mobile body 10 according to the first embodiment of the present disclosure is not limited to such an example.
  • the shape and internal structure of the autonomous mobile body 10 according to the present embodiment can be arbitrarily designed.
  • the autonomous mobile body 10 according to the present embodiment can also be realized as, for example, a walking type, a flying type, a swimming type robot, or the like.
  • FIG. 14 is a block diagram showing a configuration example of the information processing system according to the present embodiment.
  • the information processing system according to the present embodiment includes an autonomous mobile body 10, an information processing server 20, and a device to be operated 30. Further, each configuration is connected via the network 40.
  • the autonomous mobile body 10 is an information processing device that performs autonomous operation based on control by the information processing server 20.
  • the autonomous mobile body 10 according to the present embodiment can be various robots such as a traveling type, a walking type, a flying type, and a swimming type.
  • the information processing server 20 is an information processing device that controls the operation of the autonomous mobile body 10, and may be configured by, for example, a cloud or a server configured on the network 40.
  • the information processing server 20 according to the present embodiment has a function of causing the autonomous mobile body 10 to execute various incentive actions that induce communication with a user.
  • One of the features of the above-mentioned incentive movement and communication is that the behavior of the autonomous mobile body 10 in the physical space is included.
  • the operated device 30 according to the present embodiment is various devices operated by the information processing server 20 and the autonomous mobile body 10.
  • the autonomous mobile body 10 according to the present embodiment can operate various operated devices 30 based on the control by the information processing server 20.
  • the operated device 30 according to the present embodiment may be, for example, a home electric appliance such as a lighting device, a game device, or a television device.
  • the network 40 has a function of connecting each configuration included in the information processing system.
  • the network 40 may include a public line network such as the Internet, a telephone line network, a satellite communication network, various LANs (Local Area Network) including Ethernet (registered trademark), and a WAN (Wide Area Network). Further, the network 40 may include a dedicated line network such as IP-VPN (Internet Protocol-Virtual Private Network). Further, the network 40 may include a wireless communication network such as Wi-Fi (registered trademark) and Bluetooth (registered trademark).
  • the system configuration example according to the first embodiment of the present disclosure has been described above.
  • the above configuration described with reference to FIG. 14 is merely an example, and the configuration of the information processing system according to the first embodiment of the present disclosure is not limited to such an example.
  • the control function of the information processing server 20 may be implemented as a function of the autonomous mobile body 10.
  • the system configuration according to the first embodiment of the present disclosure can be flexibly modified according to specifications and operations.
  • FIG. 15 is a block diagram showing a functional configuration example of the autonomous mobile body 10 according to the present embodiment.
  • the autonomous mobile body 10 according to the present embodiment includes a sensor unit 110, an input unit 120, a light source 130, a voice output unit 140, a drive unit 150, a control unit 160, and a communication unit 170.
  • the sensor unit 110 has a function of collecting various sensor information related to the user and the surroundings.
  • the sensor unit 110 according to the present embodiment includes, for example, the above-mentioned camera 515, ToF sensor 520, microphone 540, inertial sensor 525, and the like.
  • the sensor unit 110 may include various sensors such as various optical sensors including a geomagnetic sensor, a touch sensor, an infrared sensor, etc., a temperature sensor, a humidity sensor, and the like.
  • the input unit 120 has a function of detecting a physical input operation by the user.
  • the input unit 120 according to the present embodiment includes, for example, a button such as a power switch 560.
  • the light source 130 according to the present embodiment expresses the eye movement of the autonomous moving body 10.
  • the light source 130 according to the present embodiment includes two eye portions 510.
  • the voice output unit 140 has a function of outputting various sounds including voice.
  • the audio output unit 140 according to the present embodiment includes a speaker 535, an amplifier, and the like.
  • the drive unit 150 expresses the body movement of the autonomous mobile body 10.
  • the drive unit 150 according to the present embodiment includes two wheels 570 and a plurality of motors 565.
  • Control unit 160 The control unit 160 according to the present embodiment has a function of controlling each configuration included in the autonomous mobile body 10.
  • the control unit 160 controls, for example, the start and stop of each configuration. Further, the control unit 160 inputs the control signal generated by the information processing server 20 to the light source 130, the voice output unit 140, and the drive unit 150. Further, the control unit 160 according to the present embodiment may have the same function as the operation control unit 230 of the information processing server 20 described later.
  • the communication unit 170 performs information communication with the information processing server 20, the operated device 30, and other external devices.
  • the communication unit 170 according to the present embodiment includes a connection terminal 555 and a communication device 530.
  • the functional configuration example of the autonomous mobile body 10 according to the first embodiment of the present disclosure has been described above.
  • the above configuration described with reference to FIG. 15 is merely an example, and the functional configuration of the autonomous mobile body 10 according to the first embodiment of the present disclosure is not limited to such an example.
  • the autonomous mobile body 10 according to the present embodiment does not necessarily have all of the configurations shown in FIG.
  • the functional configuration of the autonomous mobile body 10 according to the present embodiment can be flexibly deformed according to the shape of the autonomous mobile body 10 and the like.
  • FIG. 16 is a block diagram showing a functional configuration example of the information processing server 20 according to the present embodiment.
  • the information processing server 20 according to the present embodiment includes a recognition unit 210, an action planning unit 220, an operation control unit 230, and a communication unit 240.
  • the recognition unit 210 has a function of performing various recognitions related to the user, the surrounding environment, and the state of the autonomous mobile body 10 based on the sensor information collected by the autonomous mobile body 10. As an example, the recognition unit 210 may perform user identification, facial expression and line-of-sight recognition, object recognition, color recognition, shape recognition, marker recognition, obstacle recognition, step recognition, brightness recognition, and the like.
  • the recognition unit 210 performs emotion recognition, word understanding, sound source localization, etc. related to the user's voice.
  • the recognition unit 210 can recognize the ambient temperature, the existence of the animal body, the posture of the autonomous moving body 10, and the like.
  • the recognition unit 210 has a function of estimating and understanding the surrounding environment and the situation in which the autonomous mobile body 10 is placed based on the recognized information. At this time, the recognition unit 210 may comprehensively estimate the situation by using the environmental knowledge stored in advance.
  • the action planning unit 220 has a function of planning the action to be performed by the autonomous mobile body 10 based on the situation estimated by the recognition unit 210 and the learning knowledge.
  • the action planning unit 220 executes the action plan by using a machine learning algorithm such as deep learning.
  • the motion control unit 230 controls the motion of the autonomous mobile body 10 based on the action plan by the action planning unit 220.
  • the motion control unit 230 may, for example, move the autonomous moving body 10 having an oblong outer shape while maintaining the forward leaning posture.
  • the above-mentioned moving motion includes a back-and-forth motion, a turning motion, a rotational motion, and the like.
  • one of the features of the motion control unit 230 according to the present embodiment is that the autonomous mobile body 10 actively executes an incentive motion that induces communication between the user and the autonomous mobile body 10.
  • the incentive motion and communication according to the present embodiment may include the physical behavior of the autonomous mobile body 10 in the physical space. The details of the incentive motion realized by the motion control unit 230 according to the present embodiment will be described later.
  • the communication unit 240 performs information communication with the autonomous mobile body 10 and the operation target. For example, the communication unit 240 receives the sensor information from the autonomous mobile body 10 and transmits a control signal related to the operation to the autonomous mobile body 10.
  • the functional configuration example of the information processing server 20 according to the first embodiment of the present disclosure has been described above.
  • the above configuration described with reference to FIG. 16 is merely an example, and the functional configuration of the information processing server 20 according to the first embodiment of the present disclosure is not limited to such an example.
  • various functions of the information processing server 20 may be distributed and realized by a plurality of devices.
  • the function of the information processing server 20 may be realized as a function of the autonomous mobile body 10.
  • the functional configuration of the information processing server 20 according to the present embodiment can be flexibly modified according to specifications and operations.
  • the incentive motion of the autonomous mobile body 10 realized by the motion control unit 230 according to the present embodiment will be described with reference to specific examples.
  • the autonomous mobile body 10 according to the present embodiment can actively execute various incentive actions based on the control by the motion control unit 230.
  • the autonomous mobile body 10 according to the present embodiment can work on the user more impressively and activate communication by performing an incentive action accompanied by a physical behavior.
  • the incentive action according to the present embodiment may be, for example, an action for causing the user to perform a predetermined action.
  • 17 to 20 are diagrams showing an example of an incentive action for causing a user to perform a predetermined action.
  • FIG. 17 shows an example of a case where the autonomous mobile body 10 performs an incentive action to encourage the user to wake up.
  • the motion control unit 230 can cause the autonomous mobile body 10 to execute an incentive action for urging the user U1 to wake up, for example, based on the daily wake-up habit of the user and the schedule of the day of the user.
  • the motion control unit 230 outputs the voice utterance SO1 such as "Morning, get up!, The alarm sound, or the BGM to the autonomous mobile body 10.
  • the incentive action according to the present embodiment includes an incentive for voice communication.
  • the motion control unit 230 according to the present embodiment intentionally limits the number of words of the voice to be output to the autonomous mobile body 10 (in one word) or makes them in no particular order to express loveliness and hatefulness. You may.
  • the fluency of the voice of the autonomous mobile body 10 may be improved with learning, or may be designed to speak fluently from the beginning. It may also be changed based on the user's settings.
  • the motion control unit 230 executes an incentive action to escape from the user U1 on the autonomous mobile body 10 so as to hinder the stop motion. You may let me.
  • the motion control unit 230 and the autonomous mobile body 10 according to the present embodiment there is a deeper depth accompanied by physical motion, unlike the case where the alarm sound is simply passively output at the set time. It is possible to realize continuous communication.
  • FIG. 18 shows an example in which the autonomous mobile body 10 performs an incentive action for urging the user U1 to stop binge eating.
  • the incentive action for causing the predetermined action may include an action for stopping the predetermined action.
  • the motion control unit 230 outputs a voice utterance SO2 such as "overeating, gaining weight, and no use", and causes the autonomous mobile body 10 to perform an incentive action to run around on the table.
  • a warning accompanied by a physical motion is compared with a case where a warning for a health state or the like based on image recognition or the like is simply passively given by voice.
  • a warning for a health state or the like based on image recognition or the like is simply passively given by voice.
  • the incentive action as shown in the figure, a user who feels annoyed by the incentive action invites further communication such as trying to stop the incentive action or complaining about the autonomous mobile body 10. The effect is also expected.
  • FIG. 19 shows an example in which the autonomous mobile body 10 provides the user U1 with sale information and performs an incentive action to guide the user to the sale.
  • the information processing server 20 can make the autonomous mobile body 10 execute various information presentations based on the store information and event information collected from the network, the user's preference, and the like. ..
  • the motion control unit 230 outputs the voice utterance SO3 "sale, profit, let's go" to the autonomous mobile body 10, and outputs the sail information to the operated device 30 possessed by the user U1. Display. At this time, the operation control unit 230 may directly control the sail information displayed on the operated device 30, or the control unit 160 of the autonomous mobile body 10 may execute the control via the communication unit 170.
  • the motion control unit 230 outputs the voice utterance SO3 to the autonomous mobile body 10, and causes the autonomous mobile body 10 to execute an incentive action including a jump.
  • the autonomous mobile body 10 according to the present embodiment can realize a jumping operation by vigorously projecting the wheels 570 to the outside.
  • the recommendation information is recommended with physical motion as compared with the case where the recommendation information is simply provided by voice or visual information. It can give a deeper impression to the user and enhance the effect of providing information.
  • the motion control unit 230 may cause the autonomous mobile body 10 to output a voice utterance such as "take me, go with me".
  • the autonomous mobile body 10 according to the present embodiment has a size and weight that can be easily lifted by a user with one hand, and can be formed in a size that can be stored in, for example, a PET bottle holder provided in a vehicle. Therefore, the user can easily take the autonomous mobile body 10 to the outside. Further, for example, while moving in a vehicle, the motion control unit 230 can improve the convenience of the user by causing the autonomous moving body 10 to perform navigation to the destination.
  • FIG. 20 shows an example in which the autonomous mobile body 10 performs an incentive action for urging the user U1 to continue talking.
  • the motion control unit 230 controls the drive unit 150 of the autonomous mobile body 10 and repeats the forward tilting motion and the backward tilting motion to express a nod (aizuchi). Further, at this time, the motion control unit 230 appeals that the user U1 is listening to the utterance by outputting the voice utterance SO4 using the word included in the user utterance UO1 to the autonomous mobile body 10. You may.
  • the information processing server 20 may cause the autonomous mobile body 10 to perform the above-mentioned incentive operation.
  • the motion control unit 230 brings the autonomous mobile body 10 closer to the user U1 and outputs voice utterances such as "what happened?" Or "talk, listen” to the user U1. It can give U1 a chance to talk.
  • the motion control unit 230 and the autonomous mobile body 10 it is possible to interact with the user as a more familiar and friendly conversation partner as compared with the case where the user simply responds to the utterance of the user. It is possible to realize deep and continuous communication.
  • the incentive action according to the present embodiment may include an action for causing the user to perform a joint action with the autonomous mobile body 10.
  • the above-mentioned joint action includes, for example, a game by a user and an autonomous mobile body 10. That is, the motion control unit 230 according to the present embodiment can cause the autonomous mobile body 10 to execute an incentive action that invites the user to the game.
  • 21 to 24 are diagrams showing an example of an incentive action that induces a joint action between the user and the autonomous mobile body 10 according to the present embodiment.
  • FIG. 21 shows an example of a case where the autonomous mobile body 10 plays an associative game with the user U2.
  • the game targeted by the autonomous mobile body 10 for the incentive action may include a game using a language.
  • a game using a language in addition to the associative game shown in FIG. 21, "Shiritori" in the Japanese-speaking area (corresponding to "Word Chain” in the English-speaking area) and the phrase indicated by the user's gesture are autonomously moved. Examples include word guessing games (Charades) that the body 10 answers.
  • Charades word guessing games
  • the motion control unit 230 may cause the autonomous mobile body 10 to explicitly invite the game using voice utterances, but suddenly the game is unilaterally started based on the user's utterances. Then, the user may be invited to participate in the game.
  • the motion control unit 230 is an associative game using "yellow” included in the utterance based on the user utterance UO2 that "yellow flowers have bloomed" uttered by the user U2.
  • the voice utterance SO5 related to the start of is output to the autonomous mobile body 10.
  • FIG. 22 shows an example in which the autonomous mobile body 10 performs "Daruma-san fell” (corresponding to "Red light / Green Light” or “Statues”) with the user U2.
  • the games for which the autonomous mobile body 10 is the target of the incentive movement include the user and the game that requires the physical movement of the autonomous mobile body 10.
  • the autonomous mobile body 10 can move forward and turn around by having two wheels 570, and can play a game such as "Daruma-san fell" with the user. It is possible.
  • the recognition unit 210 of the information processing server 20 can recognize the user's turning action by detecting the user's face included in the image captured by the autonomous mobile body 10. Further, the recognition unit 210 may recognize the user's turning action from the user utterances UO3 and UO4 and the like.
  • the action planning unit 220 plans an action to stop on the spot, an action to dare to fall forward, etc. based on the recognition of the turning action, and the action control unit 230 autonomously moves based on the plan.
  • the drive unit 150 of 10 is controlled.
  • the autonomous mobile body 10 according to the present embodiment can recover from the fallen state by itself by incorporating a pendulum or the like.
  • the motion control unit 230 may induce the user to participate in the game by suddenly starting the game unilaterally, as in the case of the associative game.
  • the information processing server 20 stops the operation of the autonomous moving body 10 when the user's line of sight is directed toward the autonomous moving body 10, and causes the user to approach the user when the user's line of sight is off. By repeating the control, it is possible to induce the user to the game.
  • FIG. 23 shows an example in which the autonomous mobile body 10 performs "Kakurenbo" (corresponding to "Hide and seek") with the user U2.
  • the motion control unit 230 outputs an eerie BGM to the autonomous mobile body 10 together with the voice utterance SO6 indicating that the user U2 is being searched for. According to such control, it is possible to effectively express the realism of the autonomous mobile body 10 gradually approaching the user U2 and realize deeper communication.
  • the information processing server 20 causes the user U2 to the autonomous mobile body 10 by, for example, performing a SLAM map generated in advance, sound information collected when the user U2 escapes, and sound source localization related to noise generated in the surroundings. It is possible to search for.
  • FIG. 24 shows an example in which the autonomous mobile body 10 plays a computer game with the user U2.
  • the game for which the autonomous mobile body 10 according to the present embodiment is the target of the incentive action may include a computer game.
  • the motion control unit 230 may cause the autonomous mobile body 10 to execute an operation of arbitrarily activating the operated device 30 which is a game device. In this way, the motion control unit 230 can cause the autonomous mobile body 10 to perform an unintended or unintended motion of the user, that is, a mischievous motion.
  • the above-mentioned mischief includes, for example, the operation of the operated device 30 as shown in the figure.
  • the motion control unit 230 may cause the autonomous mobile body 10 to perform an action from the standpoint of the character in the game in which the user U2 is competing.
  • the motion control unit 230 may cause the autonomous mobile body 10 to behave as if the autonomous mobile body 10 actually controls the movement of the character.
  • the user U2 can strongly remind the user U2 of the feeling of competing with the autonomous mobile body 10 in a computer game, and can recognize the autonomous mobile body 10 as a more familiar existence than just a robot.
  • the motion control unit 230 causes the autonomous mobile body 10 to perform an action (such as ramming, running around, trembling, etc.) that interferes with the user U2.
  • the voice utterance SO7 corresponding to the operation may be output. According to the above operation control, it is possible to realize closer communication with the user through the computer game.
  • the motion control unit 230 causes the autonomous mobile body 10 to actively execute incentive actions related to various games so that the autonomous mobile body 10 and the user can interact with each other. Communication can be activated.
  • FIG. 25 is a diagram for explaining an incentive operation related to the presentation of the article position according to the present embodiment.
  • FIG. 25 shows an example in which the autonomous mobile body 10 according to the present embodiment performs an incentive operation indicating the position of the smartphone that the user is looking for.
  • the motion control unit 230 autonomously performs incentive actions such as lightly hitting the smartphone, performing back-and-forth movement around the smartphone, and jumping. You may let 10.
  • the motion control unit 230 when it is estimated from the user utterance UO5 that the user is searching for a predetermined article, the motion control unit 230 according to the present embodiment causes the autonomous mobile body 10 to perform an motion indicating the position of the article. Can be executed. At this time, the motion control unit 230 can effectively present information to the user by causing the autonomous mobile body 10 to perform an incentive motion near the place where the article is actually located.
  • the recognition unit 210 may detect the position of the article based on the image information registered in advance, or may detect the position based on a tag attached to the article or the like.
  • FIG. 26 is a diagram for explaining an incentive action for inducing the user to sleep according to the present embodiment.
  • FIG. 26 shows an example of a case where the autonomous mobile body 10 reads aloud to put the user U2 to sleep.
  • the autonomous mobile body 10 can read aloud, for example, a story registered as data in advance and various stories acquired via communication.
  • the motion control unit 230 normally sets a limit on the language (for example, the number of words and the vocabulary) used by the autonomous mobile body 10, the movement control unit 230 releases the limit when reading aloud. You can.
  • the motion control unit 230 may cause the autonomous mobile body 10 to expressly reproduce the voice of the character in the story, or to output sound effects, BGM, and the like together. Further, the motion control unit 230 may cause the autonomous mobile body 10 to perform an motion according to a dialogue or a scene.
  • the motion control unit 230 can control a plurality of autonomous mobile bodies 10 to read aloud and reproduce the story.
  • the motion control unit 230 causes the two autonomous moving bodies 10a and 10b to play the two characters in the story, respectively.
  • the motion control unit 230 may cause the autonomous mobile body 10 to perform control to turn off the operated device 30 which is a lighting device based on the start of sleep of the user.
  • the information processing server 20 and the autonomous mobile body 10 according to the present embodiment can realize flexible operations according to changes in the situation relating to the user and the surrounding environment.
  • the incentive action according to the present embodiment may be communication between the autonomous mobile body 10 and another device.
  • 27 and 28 are diagrams for explaining communication between the autonomous mobile body 10 and other devices according to the present embodiment.
  • FIG. 27 shows an example in which the autonomous mobile body 10 performs interpretation between the user and another device 50 which is a dog-shaped autonomous mobile body.
  • the motion control unit 230 presents information related to the internal state of the other device 50 to the user by using the voice utterance SO11.
  • the other device 50 which is a dog-shaped autonomous mobile body, may be a device that does not have a means of communication by language.
  • the motion control unit 230 can show the user information regarding the internal state of the other device 50 via the autonomous mobile body 10. According to the above-mentioned function of the motion control unit 230 according to the present embodiment, it is possible to notify the user of various information related to another device 50 which does not have a direct communication means using a language to the user. In addition, it is possible to activate communication between the user, the autonomous mobile body 10, and the other device 50 via the notification.
  • FIG. 28 shows an example of communication between the plurality of autonomous mobile bodies 10a and 10b and another device 50 which is an agent device having a projection function.
  • the motion control unit 230 controls the autonomous mobile bodies 10a and 10b and the other device 50 so that the robots communicate with each other, for example, the autonomous mobile bodies 10a and 10b and the other device 50. be able to.
  • the motion control unit 230 projects the visual information VI1 on the other device 50 via the autonomous mobile body 10. Further, the motion control unit 230 causes the autonomous mobile body 10a to output a voice utterance 12, causes the autonomous mobile body 10 to output a laughing voice, and executes an operation of shaking the main body.
  • the motion control unit 230 may execute communication between the devices using a pseudo language that the user cannot understand. According to such control, it is possible to strongly attract the interest of the user by reminding the user of a situation in which a mysterious conversation is taking place between the devices. Further, according to the control, for example, even when the other device 50 is a display device having no agent function, the user is reminded of the feeling that the display device has a personality, and the display is performed. It is expected to have the effect of improving the user's attachment to the device.
  • the motion control unit 230 causes the autonomous mobile body 10 to perform an operation of swinging the main body, but the motion control unit 230 according to the present embodiment intentionally makes the posture control unstable. By doing so, it is possible to vibrate the autonomous moving body 10. According to this control, it is possible to express emotions such as tremors, laughter, and fear without providing a separate piezoelectric element or the like.
  • the autonomous mobile body 10 can hear the utterances of people including the user. Further, the autonomous mobile body 10 expresses emotions by onomatopoeia or the like without using words. The autonomous moving body 10 can detect a step and avoid falling, but it is easy to hit an object and fall easily. In addition, when the vehicle falls, the autonomous moving body 10 cannot return to the standing position by itself. The autonomous mobile body 10 continues to act until the battery runs out, and the emotion is unstable. The autonomous mobile body 10 often trembles and gets angry, blinks a lot, and changes its eye color frequently.
  • the autonomous mobile body 10 will memorize and repeat the words of the user when the predetermined condition (for example, the number of detections) is satisfied while returning the words of the user who heard the words. Also, the autonomous mobile body 10 will be able to move without hitting an object, and will remember to ask for help if it falls. Further, the autonomous mobile body 10 expresses that it is hungry when the battery is depleted.
  • the predetermined condition for example, the number of detections
  • the autonomous mobile body 10 understands its own name by being repeatedly called by the user.
  • the autonomous mobile body 10 recognizes the user's face and shape, and remembers the user's name when a predetermined condition (for example, the number of recognitions) is satisfied.
  • the autonomous mobile body 10 ranks the reliability of recognized humans and objects. At this time, in addition to users, animals such as pets, toys, devices, etc. may be added to the higher ranks.
  • the autonomous mobile body 10 finds the charging stand, it may remember to return to the charging stand and supply power.
  • the autonomous mobile body 10 can combine known words with the memorized proper nouns and emit a short sentence (for example, "Kazuo, Genki"). Further, when the autonomous mobile body 10 recognizes a person, it tries to approach the person. Further, the autonomous mobile body 10 may be able to travel quickly.
  • the autonomous mobile body 10 tries to imitate the movement of a person or an object (for example, dance). Further, the autonomous mobile body 10 tries to imitate the special sound (siren, alarm, engine sound, etc.) heard. At this time, the autonomous mobile body 10 may reproduce similar sounds registered as data. In addition, the autonomous mobile body 10 can remember the time cycle of one day, grasp the schedule of the day, and notify the user (for example, "Kazuo, get up", "Kazuo, return", etc.).
  • the autonomous mobile body 10 can control the operation (for example, ON / OFF) of the registered device. Further, the autonomous mobile body 10 can also perform the above control based on the request of the user. The autonomous mobile body 10 can output the registered music according to the situation. The autonomous mobile body 10 can memorize a time cycle of one week, grasp the schedule of the week, and notify the user (for example, "Kazuo, burning garbage, did you put it out?").
  • the autonomous mobile body 10 remembers movements that express emotions.
  • the above expressions include movements related to emotions, such as laughter and crying.
  • the autonomous mobile 10 can memorize a time cycle of one month, grasp the schedule of the month, and notify the user (for example, "Kazuo, today, payday!).
  • Level 110-139 When the user is laughing, the autonomous mobile body 10 laughs together, and when the user is crying, he / she approaches the side and becomes concerned.
  • the autonomous mobile body 10 acquires various conversation modes, such as memorizing an aizuchi and devoting itself to listening.
  • the autonomous mobile body 10 can memorize a time cycle of one year, grasp the schedule of the year, and notify the user.
  • the autonomous moving body 10 remembers the return from the fallen state by itself and the jump during running. In addition, the autonomous mobile body 10 can play with the user by "Daruma-san has fallen” or "Hide and seek”.
  • the autonomous mobile body 10 will perform mischief by operating the registered device regardless of the user's intention. In addition, the autonomous mobile body 10 becomes scolded when scolded by the user (puberty). The autonomous mobile body 10 can grasp the position of the registered article and notify the user.
  • the autonomous mobile body 10 will be able to read aloud the story. It also has a payment function for purchasing products via a network.
  • the above is an example of the growth of the autonomous mobile body 10 according to the present embodiment.
  • the above is just an example, and the operation of the autonomous mobile body 10 can be appropriately adjusted by setting by the user or the like.
  • FIG. 29 is a flowchart showing a flow of control of the autonomous mobile body 10 by the information processing server 20 according to the present embodiment.
  • the communication unit 240 receives the sensor information from the autonomous mobile body 10 (S1101).
  • the recognition unit 210 executes various recognition processes based on the sensor information received in step S1101 (S1102), and estimates the situation (S1103).
  • the action planning unit 220 performs an action plan based on the situation estimated in step S1103 (S1104).
  • the motion control unit 230 controls the motion of the autonomous mobile body 10 based on the action plan determined in step S1104 (S1105).
  • FIG. 30 is a flowchart showing an example of the flow from the recognition process to the operation control according to the present embodiment.
  • the recognition unit 210 identifies the user based on an image captured by the autonomous mobile body 10 (S1201).
  • the recognition unit 210 performs voice recognition and intention interpretation related to the user's utterance collected by the autonomous mobile body 10 to understand the user's utterance intention (S1202).
  • the action planning unit 220 plans to approach the user, and the motion control unit 230 controls the driving unit 150 of the autonomous mobile body 10 based on the plan to bring the autonomous mobile body 10 closer to the user (S1203). ..
  • the motion control unit 230 requests based on the action plan determined by the action planning unit 220. (S1205).
  • the response behavior includes, for example, presentation of a response to an inquiry from a user, control of the operated device 30, and the like.
  • step S1202 when the user's utterance intention understood in step S1202 is not a request to the autonomous mobile body 10 (S1204: NO), the motion control unit 230 responds to the situation based on the action plan determined by the action planning unit 220.
  • the autonomous mobile body 10 is made to perform various incentive actions (S1206).
  • FIG. 31 and 32 are schematic views showing a configuration example of a sensor unit mounted on the autonomous mobile body according to the present embodiment.
  • FIG. 31 is a schematic diagram showing the sensor position when the autonomous mobile body 10 is viewed from the side
  • FIG. 32 is a schematic diagram showing the sensor position when the autonomous mobile body 10 is viewed from above.
  • the autonomous moving body 10 includes, for example, the first obstacle sensor 1101, the second obstacle sensors 1102 and 1103, and the first to fourth floor sensors 1111. -1114, a proximity sensor 1121, and a torque sensor 1122.
  • the autonomous mobile body 10 also includes an inertial sensor 525 and a camera 515, as in the above-described embodiment.
  • the first obstacle sensor 1101 is provided, for example, in front of the autonomous moving body 10 in a standing position, and detects an object such as an obstacle or a person existing in a relatively wide range in front of the front of the autonomous moving body 10. To do.
  • the first obstacle sensor 1101 may be, for example, a millimeter wave radar sensor.
  • the present invention is not limited to this, for example, a three-dimensional ToF sensor that detects the distance to an object and its shape by using reflected light, a ToF sensor that uses an infrared light source or a near-infrared light source as a light source, and the like.
  • the first obstacle sensor 1101 is a variety of sensors capable of detecting an object such as an obstacle or a person, such as an ultrasonic sensor that emits ultrasonic waves and detects the distance from the reflection to the object, and a camera that images the object. It is also possible to apply to.
  • the second obstacle sensor 1102 is provided, for example, on the right side of the front of the autonomous moving body 10 in a standing state, and detects an object such as an obstacle or a person existing on the right front of the autonomous moving body 10.
  • the second obstacle sensor 1103 is provided, for example, on the left side of the front of the autonomous moving body 10 in a standing state, and detects an object such as an obstacle or a person existing on the left front of the autonomous moving body 10.
  • These second obstacle sensors 1102 and 1103 may be, for example, one-dimensional ToF sensors that measure the distance to an object existing in one direction.
  • the present invention is not limited to this, and various sensors capable of detecting objects such as obstacles and people, such as millimeter-wave radar sensors, three-dimensional ToF sensors, and ultrasonic sensors, are applied to the second obstacle sensors 1102 and 1103. Is also possible.
  • the detection range of the first obstacle sensor 1101 and the detection range of the second obstacle sensor 1102 or 1103 overlap. That is, in the present embodiment, the object existing on the right front side of the autonomous moving body 10 is detected by the first obstacle sensor 1101 and the second obstacle sensor 1102, and the object existing on the left front side of the autonomous moving body 10 is detected. , The first obstacle sensor 1101 and the second obstacle sensor 1103 are configured to detect.
  • the first to fourth floor surface sensors 1111 to 1114 are arranged so as to be arranged along the outer circumference of the autonomous moving body 10 in a standing state, and detect the shape of the floor surface around the autonomous moving body 10.
  • the floor surface on which the autonomous moving body 10 is placed may be, for example, a floor surface such as flooring or folding, or an upper surface of a top plate such as a table or desk, and the shape thereof is the shape of the outer edge of the upper surface of the top plate. Or, it may be the shape of a room or a corridor separated by a wall, a rail, or the like.
  • the outer edge of the upper surface of the top plate, walls, rails, etc. that separate rooms, corridors, etc. are referred to as "boundaries".
  • the first floor surface sensor 1111 is provided diagonally downward to the right of the front of the autonomous moving body 10 in a standing state, and detects the boundary on the right front of the autonomous moving body 10.
  • the second floor surface sensor 1112 is provided diagonally downward to the left of the front of the autonomous moving body 10 in the standing state, detects the boundary of the left front of the autonomous moving body 10, and detects the boundary of the left front of the autonomous moving body 10.
  • the 1113 is provided diagonally downward to the right of the rear surface of the autonomous moving body 10 in the standing state, detects the boundary on the right rear side of the autonomous moving body 10, and the fourth floor surface sensor 1114 is in the standing state.
  • the installation interval on the outer periphery of the autonomous moving body 10 of each of the first to fourth floor surface sensors 1111 to 1114 may be, for example, a 90 ° interval.
  • first to fourth floor surface sensors 1111 to 1114 may be, for example, one-dimensional ToF sensors.
  • the present invention is not limited to this, and any sensor such as an ultrasonic sensor or a proximity sensor that can detect the distance to an object (floor surface, etc.) diagonally downward to which each sensor faces, or a sensor that can specify the shape of the boundary.
  • various sensors can be applied to the first to fourth floor surface sensors 1111 to 1114.
  • the proximity sensor 1121 is provided on the bottom of the autonomous moving body 10 in a standing or sitting state, for example, and detects whether or not an object such as a floor is close to the bottom of the autonomous moving body 10. That is, in the proximity sensor 1121, the autonomous moving body 10 is placed in a standing or sitting position with respect to the floor surface or the like, or is lifted by the user or the like, or is in a horizontal position with respect to the floor surface or the like. Detects whether or not it is placed in a state. Instead of the proximity sensor 1121, a sensor such as a ToF sensor that can determine whether or not an object such as a floor surface is close to the bottom of the autonomous moving body 10 may be used.
  • a sensor such as a ToF sensor that can determine whether or not an object such as a floor surface is close to the bottom of the autonomous moving body 10 may be used.
  • the torque sensor 1122 is provided on the axle of the wheel 570, for example, and detects the torque generated on the axle.
  • various torque sensors such as a magnetostrictive type, a strain gauge type, a piezoelectric type, an optical type, a spring type, and a capacitance type may be adopted.
  • the inertial sensor 525 may be a sensor capable of detecting at least one of acceleration, angle, angular velocity, angular acceleration, etc., such as IMU (Inertial Measurement Unit). ..
  • the camera 515 is an imaging device that captures images of the user and the surrounding environment.
  • the image data acquired by the camera 515 may be provided to the user as a photograph, used for face recognition of the user, or the like.
  • the sensor unit 110 includes various sensors such as a microphone for inputting a sound such as a voice emitted by a user and a GPS (Global Positioning System) for measuring the position of the autonomous moving body 10. It can be.
  • a microphone for inputting a sound such as a voice emitted by a user
  • a GPS Global Positioning System
  • control operation described below may be executed by the control unit 160 in the autonomous mobile body 10 or by the operation control unit 230 in the information processing server 20.
  • the operation control unit 230 in the information processing server 20 executes the information processing server 20 will be described as an example.
  • the collision prevention operation is an operation for the autonomous moving body 10 to avoid obstacles existing in the traveling direction or the traveling route to prevent a collision.
  • This collision prevention operation includes, for example, an obstacle detection operation and an obstacle avoidance operation, and an obstacle is used by using the first obstacle sensor 1101 in the sensor unit 110 and the second obstacle sensors 1102 and 1103. Detection and avoidance are performed.
  • the first obstacle sensor 1101 and the second obstacle sensors 1102 and 1103 are different types of sensors.
  • a millimeter-wave radar sensor is used for the first obstacle sensor 1101
  • a different type of sensor from the millimeter-wave radar sensor such as a one-dimensional ToF sensor is used for the second obstacle sensors 1102 and 1103. Is preferable.
  • the motion control unit 230 advances the autonomous moving body 10 from the detection result obtained by the first obstacle sensor 1101 and the detection result obtained by the second obstacle sensor 1102 and / or 1103. Determine if there is an obstacle in the direction or on the travel route.
  • the motion control unit 230 moves the traveling direction or the traveling route of the autonomous moving body 10. Determine that there is an obstacle above.
  • the first obstacle sensor 1101 and the second obstacle sensors 1102 and 1103 it is possible to detect obstacles more reliably.
  • millimeter-wave radar sensors, one-dimensional ToF sensors, ultrasonic sensors, etc. have different detection accuracy depending on the size, shape, material, color, etc. of the object, and also have detection conditions such as scratches and dust adhesion. Robustness to change is also different.
  • a millimeter-wave radar sensor has low detection accuracy for a transparent object
  • a range-finding sensor such as a one-dimensional ToF sensor has a significantly reduced detection accuracy due to scratches or dust on the sensor window. Therefore, different types of sensors are used for the first obstacle sensor 1101 and the second obstacle sensors 1102 and 1103, and when an obstacle is detected by any of the sensors, it is determined that an obstacle exists. It is possible to improve the robustness to the type of the object to be detected, the detection conditions, and the like.
  • the motion control unit 230 executes an operation for avoiding the obstacle (obstacle avoidance operation). Specifically, the motion control unit 230 determines whether to avoid the obstacle to the right side or the left side based on the traveling direction and / or the traveling route of the autonomous moving body 10 and the position of the obstacle, and moves in the determined direction. Advance the autonomous mobile body 10.
  • the motion control unit 230 executes an obstacle avoidance operation or executes an obstacle avoidance operation by updating the travel route so that the travel route to the destination includes a route for avoiding obstacles.
  • the obstacle avoidance operation may be realized by various methods such as newly determining a traveling route to the destination based on the current position of the autonomous moving body 10 after avoiding the obstacle. Further, the obstacle avoidance operation may include deceleration or stop of the autonomous moving body 10.
  • the detection results obtained by the second obstacle sensors 1102 and 1103 may be used to determine whether to avoid the obstacle to the right side or the left side. For example, when an obstacle is detected by the second obstacle sensor 1102 arranged on the right front surface, the motion control unit 230 passes on the left side of the obstacle and the second obstacle sensor arranged on the left front surface. If an obstacle is detected in 1103, the autonomous moving body 10 may be controlled so as to pass on the right side of the obstacle.
  • the fall collision prevention operation is an operation for preventing the autonomous moving body 10 from falling from a table or the like or colliding with a wall.
  • This fall collision prevention operation includes, for example, a boundary detection operation and a boundary avoidance operation, and the boundary is detected and avoided by using a plurality of (four in this example) first to fourth floor sensors 1111 to 1114. Is executed.
  • the first to fourth floor surface sensors 1111 to 1114 are arranged along the outer circumference of the autonomous moving body 10 in the standing position, for example. Therefore, by using the first to fourth floor surface sensors 1111 to 1114, it is possible to detect the shape of the boundary around the autonomous moving body 10. As a result, it is possible to prevent the autonomous moving body 10 from falling from a table or the like or colliding with a wall regardless of the direction in which the autonomous moving body 10 travels.
  • the detection accuracy is greatly reduced due to scratches on the sensor window and adhesion of dust.
  • the motion control unit 230 monitors the amount of change (differential value) of the value detected by each of the first to fourth floor surface sensors 1111 to 1114, and the first to fourth floor surface sensors 1111. From both the distance to the boundary obtained from the absolute value of the value detected in each of 1114 and the amount of change in the value detected in each of the first to fourth floor sensors 1111 to 1114, in each direction. Estimate or specify the position and distance of the boundary.
  • the one-dimensional ToF sensors and the like used in the first to fourth floor surface sensors 1111 to 1114 have solid values, and the distance measurement accuracy thereof is generally different for each solid.
  • the first to fourth floor surface sensors 1111 to 1114 are calibrated before the autonomous moving body 10 is shipped or when the autonomous moving body 10 is initially started. This makes it possible to improve the robustness of distance measurement and detection accuracy.
  • the number of floor surface sensors 1111 to 1114 is not limited to four, and various changes are made. May be done. At that time, by determining the number of floor surface sensors in consideration of the expansion of the detection range of each floor surface sensor, it is possible to accurately detect the existence of a boundary around the autonomous moving body 10 and the distance to the boundary. It becomes.
  • the operation control unit 230 executes an operation for avoiding the boundary (boundary avoidance operation). Specifically, the motion control unit 230 determines whether to change the traveling direction to the right side or the left side based on the traveling direction and / or the traveling route and the position of the boundary of the autonomous moving body 10, and determines the determined direction. The direction of travel of the autonomous mobile body 10 is corrected.
  • the motion control unit 230 executes the boundary avoidance operation by updating the travel route so that the travel route to the destination includes the route avoiding the boundary, or executes the boundary avoidance operation to set the boundary.
  • the boundary avoidance operation may be realized by various methods such as newly determining a traveling route to the destination based on the current position of the autonomous moving body 10 after avoiding. Further, the boundary avoidance operation may include deceleration or stop of the autonomous moving body 10.
  • the anti-slip operation prevents the wheels 570 from idling when the autonomous moving body 10 is lifted by a user or the like, falls from a table or the like, or rolls over. It is an operation to do.
  • This slip prevention operation includes, for example, a lifting or the like detection operation and a wheel stop operation, and the proximity sensor 1121, the torque sensor 1122, and the inertial sensor 525 are used to detect the lifting or the like of the autonomous moving body 10 and the wheels. 570 stops and are executed.
  • the first to fourth floor surface sensors 1111 to 1114 may be further used for the lifting and the like detection operation.
  • the motion control unit 230 further controls the wheel 570 when, for example, at least one of the following conditions (1) to (4) or a predetermined number or more of the following conditions (1) to (4) are satisfied while the autonomous moving body 10 is traveling. To prevent slipping, the rotation of the wheel 570 is stopped.
  • (1) When the proximity sensor 1121 detects that the bottom of the autonomous moving body 10 is separated from the floor surface (2)
  • the inertial sensor 525 detects an acceleration change in a predetermined direction (for example, the Z-axis direction)
  • the values of all of the first to fourth floor sensors 1111 to 1114 change by a predetermined value or more (4)
  • the torque detected by the torque sensor 1122 does not change for a certain period of time or more
  • the wheel 570 by configuring the wheel 570 to detect the idling (or its possibility) using different types of sensors, it is possible to prevent the wheel 570 from idling more reliably. For example, if the proximity sensor 1121 is blocked by the user's hand or the like, or if the autonomous moving body 10 is slowly lifted and the inertial sensor 525 cannot detect a change in acceleration in the vertical direction (Z-axis direction), other factors may occur. Based on the detection result of the sensor, it is possible to detect the lifting, falling, falling, etc. of the autonomous moving body 10 and stop the rotation of the wheel 570.
  • the axle of the wheel 570 periodically keeps the autonomous moving body 10 in the inverted state.
  • the changing torque is given by the motor 565. Therefore, during this time, the torque sensor 1122 detects the torque that changes periodically.
  • the torque applied to the axle by the motor 565 is controlled by, for example, feedback control based on the value detected by the inertial sensor 525. Therefore, for example, when the autonomous moving body 10 is lifted, dropped, or overturned and the acceleration detected by the inertial sensor 525 is not accompanied by a periodic change, the torque detected by the torque sensor 1122 is also periodic. It will not be accompanied by a change. Therefore, in the present embodiment, if the torque detected by the torque sensor 1122 does not change for a certain period of time or more, the motion control unit 230 may determine that the autonomous moving body 10 has been lifted, dropped, or has fallen.
  • the conditions for determining that the autonomous moving body 10 has been lifted, dropped, or fallen are not limited to the above conditions.
  • the motion control unit 230 may determine that the autonomous moving body 10 has been lifted, dropped, or has fallen, regardless of other conditions.
  • the condition (2) when the acceleration change in the Z-axis direction detected by the inertial sensor 525 exceeds a preset threshold value, the motion control unit 230 raises or drops the autonomous moving body 10. It may be determined that the user has fallen.
  • the condition (3) when the amount of change of the value detected by each of the first to fourth floor surface sensors 1111 to 1114 is equal to or more than a predetermined value, the motion control unit 230 lifts the autonomous moving body 10. It may be determined that it has fallen or has fallen.
  • the condition (4) when the torque detected by the torque sensor 1122 suddenly becomes lighter, even if the motion control unit 230 determines that the autonomous moving body 10 has been lifted, dropped, or has fallen. Good.
  • the motion control unit 230 determines whether or not the autonomous moving body 10 is in an upright state (also referred to as an inverted state) based on the detection results acquired by the first to fourth floor surface sensors. May be determined. Then, when the motion control unit 230 determines that the autonomous mobile body 10 is in the inverted state based on the detection result, the motion control unit 230 sets the motor 565, which is a drive mechanism of the wheels 570, so that the autonomous mobile body 10 maintains the inverted state. You may control it.
  • a human sensation operation for detecting whether or not a user or the like is nearby and a user's breathing are performed.
  • the breathing detection operation for detecting the above, the gesture detection operation for detecting the gesture of the user or the like, and the like may be executed based on the detection result obtained by the sensor unit 110.
  • the human sensation operation may be an operation of detecting whether or not there is a person or the like around the autonomous moving body 10 and switching between the normal mode and the standby mode based on the result.
  • the respiration detection operation may be an operation of detecting the respiration of a person, a pet, or the like and specifying the health state, psychological state, or the like of the detection target based on the detection result.
  • the gesture detection action may be an action of detecting a gesture action of a person or the like and executing a reaction or an action according to the detected gesture action.
  • the first obstacle sensor 1101 may be used for these operations, or other sensors may be used.
  • FIG. 33 is a flowchart showing an example of the main operation executed by the operation control unit according to the present embodiment.
  • the operation control unit 230 first sets, for example, the destination of the autonomous mobile body 10 (step S2001).
  • the motion control unit 230 identifies the position of the user's face based on the image data acquired by the camera 515, the position of the object detected by the first obstacle sensor 1101, and the like, and is near the specified face position. Set the position on the table as the destination.
  • the operation control unit 230 determines the traveling route to the destination set in step S2001 (step S2002).
  • a technique such as SLAM (including a simplified SLAM) may be used.
  • the motion control unit 230 drives the motor 565 and the like to start traveling along the travel route of the autonomous mobile body 10 (step S2003).
  • the motion control unit 230 While traveling, the motion control unit 230 constantly or periodically monitors the detected values from the first obstacle sensor 1101, the second obstacle sensors 1102 and 1103, and the first to fourth floor surface sensors 1111 to 1114. By doing so, the detection of boundaries and obstacles is executed (step S2004). If no boundary or obstacle is detected (NO in step S2004), the motion control unit 230 proceeds to step S2007. On the other hand, when a boundary or an obstacle is detected (YES in step S2004), the motion control unit 230 executes the boundary avoidance operation or the obstacle avoidance operation (step S2005). Further, the motion control unit 230 updates the travel route to the destination by recalculating the travel route to the destination (step S2006), and proceeds to step S2007.
  • step S2007 the motion control unit 230 determines whether or not the vehicle has arrived at the destination. If the vehicle has not arrived at the destination (NO in step S2007), the operation control unit 230 returns to step S2004 and repeats the subsequent operations until it arrives at the destination. On the other hand, when arriving at the destination (YES in step S2007), the operation control unit 230 determines whether or not to end this operation (step S2008), and when it ends (YES in step S2008), performs this operation. finish. On the other hand, if this operation is not completed (NO in step S2008), the operation control unit 230 returns to step S2001 and executes the subsequent operations.
  • FIG. 34 is a flowchart showing an example of the slip prevention operation according to the present embodiment.
  • the operation control unit 230 constantly or predetermined values detected from the proximity sensor 1121, the inertial sensor 525, the first to fourth floor surface sensors 1111 to 1114, and the torque sensor 1122. By monitoring in the cycle of (step S2101), the autonomous moving body 10 is detected for lifting, falling, or falling (step S2101).
  • step S2101 When the lifting of the autonomous moving body 10 or the like is detected (YES in step S2101), the motion control unit 230 executes an action of stopping the rotation of the wheels 570 (step S2102). Even in this state, the motion control unit 230 constantly monitors the detected values from the proximity sensor 1121, the inertial sensor 525, the first to fourth floor surface sensors 1111 to 1114, and the torque sensor 1122 at regular intervals or at a predetermined cycle. ing.
  • the motion control unit 230 places the autonomous moving body 10 on the floor based on the detection values acquired by the proximity sensor 1121, the inertial sensor 525, the first to fourth floor surface sensors 1111 to 1114, and / or the torque sensor 1122. It is detected that the device is placed on a surface or a table (step S2103).
  • step S2103 When the placement of the autonomous moving body 10 on the floor surface or the like is detected (YES in step S2103), the motion control unit 230 releases the stop of the wheel 570 (step S2104). As a result, the autonomous mobile body 10 returns to a state in which it can travel. Therefore, the motion control unit 230 determines the travel route to the destination by executing the operation described with reference to FIG. 33 from the beginning, for example. The autonomous mobile body 10 is driven to the destination.
  • the operation control unit 230 determines whether or not to end the main operation (step S2105), and if it ends (YES in step S2105), ends the main operation. On the other hand, if it does not end (NO in step S2105), the operation control unit 230 returns to step S2101 and executes the subsequent operations.
  • the operation control unit 230 executes a mode switching operation for switching between the normal operation mode and the standby mode.
  • the normal operation mode is a mode for executing an interactive operation with the user, mainly the main operation shown in FIG. 33
  • the standby mode is a mode for stopping the operation of the autonomous mobile body 10 to save power. It may be a mode for attempting. Further, in this description, for simplification, it is assumed that the normal operation mode is first executed after the activation of the autonomous mobile body 10.
  • FIG. 35 is a flowchart showing an example of the mode switching operation according to the present embodiment.
  • the operation control unit 230 sets, for example, the human detection rate of the human sensory operation using the first obstacle sensor 1101 to the first human detection rate (step). S2201), the human sensory operation is executed at the set first human detection rate (step S2202).
  • the first person detection rate may be a rate necessary and sufficient for interactive communication with the user, for example, once every 0.1 seconds or once per second.
  • the operation control unit 230 shifts the operation mode of the autonomous mobile body 10 to the standby mode (step S2203). Then, the motion control unit 230 sets the human detection rate of the human sensory motion using the first obstacle sensor 1101 to, for example, a second human detection rate lower than the first human detection rate (step S2204). , The human sensory operation is executed at the set second human detection rate (step S2205).
  • the second person detection rate may be lower than the first person detection rate, for example, once every 10 seconds or once a minute.
  • step S2205 when the motion control unit 230 detects a person (YES in step S2205), the motion control unit 230 returns the motion mode of the autonomous mobile body 10 to the normal mode (step S2206), and the human feeling using the first obstacle sensor 1101.
  • the operation person detection rate is set to the first person detection rate (step S2207).
  • the operation control unit 230 determines whether or not to end the main operation (step S2208), and if it ends (YES in step S2208), ends the main operation. On the other hand, if it does not end (NO in step S2208), the operation control unit 230 returns to step S2202 and executes the subsequent operations.
  • FIG. 36 is a flowchart showing an example of the person detection rate switching operation according to the present embodiment.
  • the operation control unit 230 sets the person detection rate to the first person detection rate in the normal operation mode immediately after the activation of the autonomous mobile body 10 (step S2301).
  • This step S2301 may be the same as step S2201 in FIG.
  • the motion control unit 230 determines whether the autonomous moving body 10 is traveling or stationary based on the detection value from the torque sensor 1122, for example (step S2302). When stopped (NO in step S2302), the operation control unit 230 proceeds to step S2307.
  • the motion control unit 230 sets a third person detection rate according to the traveling speed of the autonomous moving body 10 (step S2303). For example, when the autonomous moving body 10 has a function of changing the traveling speed in four stages, a different third person detection rate may be set in advance for each stage. At that time, the higher the traveling speed, the higher the third person detection rate may be set.
  • the motion control unit 230 detects the traveling speed of the autonomous moving body 10 based on, for example, the detection value from the torque sensor 1122, and monitors whether or not there is a change in the traveling speed (step S2304).
  • the motion control unit 230 determines whether or not the autonomous moving body 10 has stopped (step S2305), and when it has not stopped (NO in step S2305). , Return to step S2303, and set a third person detection rate according to the running speed after the change.
  • the motion control unit 230 sets the first person detection rate (step S2306) and proceeds to step S2307.
  • step S2307 the operation control unit 230 determines whether or not to end this operation, and if it ends (YES in step S2307), ends this operation. On the other hand, if it does not end (NO in step S2307), the operation control unit 230 returns to step S2302 and executes the subsequent operations.
  • the autonomous mobile body 10 is derived from the first obstacle sensor 1101, the second obstacle sensors 1102 and 1103, the first to fourth floor sensors 1111 to 1114, and the like. Based on the detection result, an operation (mapping operation) of creating a map in which obstacles, boundaries, etc. existing around the self are mapped may be executed.
  • FIG. 37 is a flowchart showing an example of the mapping operation according to the present embodiment.
  • the operation control unit 230 first determines the position of the autonomous moving body 10 (hereinafter referred to as self-position) (step S2401).
  • This self-position is, for example, a position on a two-dimensional map whose starting point (origin) is the position at the time of starting the autonomous moving body 10 or the position where the autonomous moving body 10 first stops after the starting, and the direction at that time is the X-axis direction. May be good.
  • the coordinates (self-position) of the position of the autonomous moving body 10 with respect to the origin on the two-dimensional map are, for example, the moving distance of the autonomous moving body 10 detected by the encoder (or potentiometer) provided on the axle of the wheel 570 and the moving distance of the autonomous moving body 10.
  • the motion control unit 230 constantly or at a predetermined cycle monitors the detected values from the first obstacle sensor 1101, the second obstacle sensors 1102 and 1103, and the first to fourth floor surface sensors 1111 to 1114. By doing so, the detection of boundaries and obstacles is executed (step S2402). If no boundary or obstacle is detected (NO in step S2402), the operation control unit 230 proceeds to step S2407.
  • step S2402 determines whether or not the detected boundary is a boundary (step S2403), and when the boundary is detected. (YES in step S2403), the position of the detected boundary is arranged on the two-dimensional map (step S2404), and the process proceeds to step S2407.
  • step S2403 determines whether or not the detected object is an obstacle (step S2405), and the obstacle is detected. If (YES in step S2405), the position of the detected obstacle is placed on the two-dimensional map (step S2406), and the process proceeds to step S2407. If the detected object is neither a boundary nor an obstacle (NO in step S2405), the operation control unit 230 proceeds to step S2407 as it is.
  • step S2407 the operation control unit 230 determines whether or not to end this operation, and if it ends (YES in step S2407), ends this operation. On the other hand, if it does not end (NO in step S2407), the operation control unit 230 returns to step S2401 and executes the subsequent operations.
  • the two-dimensional map data may be stored in, for example, the RAM 873 or the storage 880 mounted on the autonomous mobile body 10, or may be stored in the removable recording medium 901 (see FIG. 56).
  • the autonomous mobile body 10 according to the present embodiment can operate smoothly according to the situation, and thus realizes more natural and effective communication with the user. It becomes possible.
  • the voice expression and motion expression of the autonomous mobile body 10 described above can realize more natural and effective communication with the user by coordinating with each other.
  • the "voice expression” referred to here may include, for example, utterances by the autonomous mobile body 10 and reproduction / output of whistling, onomatopoeia, breath sounds, and the like. Further, the "motion expression” may include movements such as eye movements of the autonomous moving body 10, head movements, movements of the entire autonomous moving body 10, and posture changes.
  • a speaker 535 corresponding to the mouth of the autonomous mobile body 10 may be used for voice expression.
  • the voice expression may include utterance in natural language, reproduction of BGM (Background Music), output of sound effect (SE), reproduction of onomatopoeia, reproduction of breath sounds, and the like.
  • the LED of the eye portion 510 corresponding to the eyes of the autonomous moving body 10 may be used to express the movement of the eyes.
  • the motion expression of the eye may include the color of the LED of the eye portion 510, its change, blinking, blinking, local blinking, and the like.
  • a substrate 505 corresponding to the head of the autonomous mobile body 10 may be used for the motion expression of the head.
  • the motion expression of the head may include motions realized by rotation, tilting, swinging, etc. of the substrate 505, for example, motions such as "squeezing", “nodding”, and “shaking the head”.
  • the drive unit 150 that expresses the body motion of the autonomous mobile body 10 may be used.
  • the overall motion expression includes movement of the entire autonomous moving body 10 by the wheels 570 and changes in posture, for example, “turning”, “tilting”, “swaying left and right”, “surprisingly descending”, and “trajectory”. Actions such as “moving as drawn”, “moving or moving suddenly slowing down”, “moving or moving faster”, “changing posture from standing to sitting (sitting)” can be included. ..
  • temporal coordination may mean, for example, matching the timing of the utterance and the movement
  • semantic coordination may mean, for example, to match the meaning of the utterance content with the content expressed by the action.
  • a mechanism for controlling the start timing and end timing of each based on the time required for voice expression and the time required for motion expression is required.
  • information on the situation around the autonomous mobile body 10 and the internal state of the autonomous mobile body 10 may be utilized for the coordination between the voice expression and the motion expression described below.
  • the sensor unit 110 described above can be used to acquire the situation around the autonomous mobile body 10 and the internal state of the autonomous mobile body 10.
  • the camera 515 that serves as the eyes of the autonomous mobile body 10 in the sensor unit 110, for example, a situation relating to a person, an object, a scene, a situation, a place, another autonomous mobile body, or the like around the autonomous mobile body 10. It is possible to get information about.
  • the microphone 540 that serves as the ear of the autonomous mobile body 10, for example, a natural language uttered by the owner of the autonomous mobile body 10 or a person other than the owner, a mobile phone, a smart speaker, an electric appliance, etc. It is possible to acquire information about sounds such as voices and alarms emitted from. Furthermore, by analyzing the sound obtained by the microphone 540 in more detail, the source (including the speaker) of the sound, its direction, and the loudness and nature of the sound (whether it is a voice or a noise, etc.) can be determined. It is also possible to identify.
  • the autonomous moving body can be lifted, struck, or collided. It is possible to obtain information on touches and drops.
  • first to fourth floor surface sensors 1111 to 1114 that serve as eyes in the downward direction around the autonomous moving body 10
  • a cliff existing around the autonomous moving body 10 can be used. It is possible to obtain information about boundaries.
  • the first obstacle sensor 1101 and / or the second obstacle sensor 1102 that serves as the eyes of the autonomous moving body 10 in the forward direction as described above, for example, it exists in front of the autonomous moving body 10. It is possible to obtain information about objects such as people and obstacles.
  • the autonomous mobile body 10 has information on applications such as the connection status to its own network 40, the charging status of the internal battery, and an application such as SNS (Social Networking Service) that cooperates with itself (for example, information on incoming mail or message). Etc.) may be available.
  • applications such as the connection status to its own network 40, the charging status of the internal battery, and an application such as SNS (Social Networking Service) that cooperates with itself (for example, information on incoming mail or message). Etc.) may be available.
  • applications such as the connection status to its own network 40, the charging status of the internal battery, and an application such as SNS (Social Networking Service) that cooperates with itself (for example, information on incoming mail or message). Etc.) may be available.
  • SNS Social Networking Service
  • the autonomous mobile body 10 may have information on a device-specific character, emotion, personality, etc. as a personality with respect to another autonomous mobile body 10.
  • interrupt processing occurs during communication with the user
  • a mechanism for executing interrupt processing in a more natural way while reducing the discomfort given to the user is also required. For example, if an interrupt process for moving to the charger occurs due to low battery level during communication with the user, the user is informed of the reason and then the user is moved to the charger. By taking the above, it becomes possible to execute interrupt processing in a more natural manner while reducing the discomfort given to the user. Also, for example, when interrupt processing for avoiding obstacles or cliffs occurs during communication with the user, words that people would say in the same situation (for example, "Ah” or "Abunai”). By executing interrupt processing for avoidance while issuing (such as), it is possible to execute interrupt processing in a more natural way while reducing the discomfort given to the user.
  • FIG. 38 is a diagram for explaining an example of cooperation between the voice expression and the motion expression according to the present embodiment.
  • the cooperative operation when the autonomous mobile body 10 utters "Today's weather is wonderful" is illustrated.
  • the actions executed by the autonomous mobile body 10 are, for example, “start motion”, “main motion (single / loop)", and “end motion”. It may be composed of.
  • the start motion is a "due” or “call” operation that is introduced into the main motion, and for example, a motion such as tilting the autonomous moving body 10 in the lateral direction can be applied.
  • the autonomous mobile body 10 introduces a sound or voice (hereinafter referred to as an introduction sound) that is introduced when uttering "Today's weather is wonderful", which is the text of the utterance content.
  • a filler such as "um” or a breathing sound "su" when inhaling may be emitted.
  • the main motion is the action executed when the text of the utterance content (hereinafter referred to as the main voice), "Today's weather is wonderful" is emitted.
  • This main motion may be composed of one motion (hereinafter, also referred to as a unit motion) such as swinging the body left and right, or may be composed of a continuous motion in which the same motion loops.
  • a unit motion such as swinging the body left and right
  • the main motion may consist of a single motion.
  • the main motion may consist of continuous motions that repeat the same unit motion until the output of the main voice is completed.
  • the end motion is an operation when returning from the cooperative operation to the normal operation, and for example, a motion for putting the autonomous moving body 10 into a standing state which is a normal posture can be applied. Further, during the period during which the end motion is being executed, the autonomous mobile body 10 uses a sound or voice (hereinafter referred to as an end sound) for closing the utterance, for example, a terminal word "yo" or "fu". You may sigh.
  • an end sound hereinafter referred to as an end sound
  • the voice expression and the motion expression are coordinated by the autonomous moving body 10 operating so as to output the introduction sound, the main sound, and the end sound in accordance with the start motion, the main motion, and the end motion. Is possible.
  • the motion for outputting one utterance content includes the start motion, the main motion, and the end motion, and for example, one or both of the start motion and the end motion may be omitted.
  • FIG. 39 is a block diagram showing an example of a system configuration for cooperative operation according to the present embodiment.
  • the autonomous mobile body 10 includes a body controller 360, a voice module 340, a motion module 350, a combination list 301, a motion database (DB) 302, and a motion database (DB) 302 as a system configuration for cooperative operation. It includes a file storage area 303.
  • the body controller 360 has, for example, a configuration corresponding to the control unit 160 in FIG. 15, and operates the voice module 340 and the motion module 350, respectively, based on the utterance content (text) received from the information processing server 20 and the intention.
  • the body controller 360 By creating a schedule and causing each of the voice module 340 and the motion module 350 to execute an operation according to the created schedule, an interaction in which the voice expression and the motion expression are coordinated is realized.
  • the "intention” in this description may be attribute information (also referred to as a parameter) related to attributes such as the intention (including information such as emotion) of the utterance content such as emotion (emotion).
  • This intention may be a parameter specified by the information processing server 20, or may be a concept included in a category described later. That is, in the present embodiment, a plurality of motions may be divided into different categories for each intention.
  • the voice module 340 has, for example, a configuration corresponding to the voice output unit 140 in FIG. 15, and realizes voice expression by outputting sound or voice from the speaker 535 according to the utterance schedule input from the body controller 360.
  • the motion module 350 has, for example, a configuration corresponding to the light source 130 and the drive unit 150 in FIG. 15, and according to the operation schedule input from the body controller 360, the LED of the eye unit 510 and the substrate 505 corresponding to the head portion.
  • the motion expression is realized by controlling the motor 565, the motor 565 for driving the wheels 570, the motor 565 for controlling the forward leaning posture, and the like.
  • FIG. 40 is a diagram showing an example of a combination list according to the present embodiment.
  • the combination list 301 may be, for example, a list in which combinations of text, audio files, and motion information are listed.
  • the text is a text version of the content to be spoken by the autonomous mobile body 10, for example, a character string such as "Hey” or "Hello", or the autonomous mobile body 10 set by default or set by the user. It may be a character string of the name of.
  • the audio file may be, for example, an audio file for outputting the text associated therewith by sound or audio, or a file path indicating the storage position of the audio file.
  • the audio file is a file in which audio data to be output by the voice module 340 is stored, and may be, for example, various audio files such as a WAVE file, an MP3 file, and a MIDI file.
  • the body controller 360 may generate a synthetic voice based on the text and use the generated synthetic voice as the voice file.
  • the body controller 360 does not have to execute the motion while outputting the corresponding voice file (or synthetic voice), and the motion DB 302 has a predetermined rule such as round robin. Alternatively, a randomly selected motion may be executed.
  • the motion information may be, for example, a motion ID (motion_id) or a category ID (category_id).
  • the motion ID is motion information (for example, LED light emission time, motor 565 drive amount, etc.) to be executed by the autonomous mobile body 10 when the text associated with the motion ID is output as sound or voice. It may be information for uniquely identifying a file (hereinafter, also referred to as a motion file) in which a file (also referred to as a motion file) is stored.
  • a UUID Universalally Unique Identifier
  • the motion file for example, various files such as a CVS (Comma Separated Value) file and a TXT file may be used.
  • the category ID may be information for uniquely identifying a category to which a plurality of motions belong. That is, in the present embodiment, it is possible to associate one motion with one text in a one-to-one manner, or to associate a plurality of motions with each category for one text.
  • the category ID for example, a UUID or the like may be used as in the motion ID.
  • the motion information may store an application ID (app_id) in association with the motion ID.
  • the application ID may be, for example, information for uniquely identifying the application when executing the motion.
  • the application may be an application executed by the body controller 360 when the motion is executed.
  • FIG. 41 is a diagram showing an example of a motion DB according to the present embodiment. As shown in FIG. 41, in the motion DB 302, the motion ID, the category ID, the emotion, the loop possibility flag, the end adjustment possibility flag, and the path to the motion file are associated with each other.
  • Emotion is a parameter for giving emotion to the autonomous mobile body 10 when performing voice expression or motion expression.
  • the emotion is a parameter indicating the emotion expressed by the autonomous moving body 10 when the motion expression or the voice expression is executed based on the motion file or the voice file (or the synthetic voice) associated with the emotion. ..
  • the loop enable / disable flag is a flag that specifies whether or not the motion of the motion ID associated with the loop can be repeatedly executed (looped).
  • the loop enable / disable flag is a flag for indicating whether or not the same motion may form a continuous continuous motion.
  • the end matching flag is an audio file (or an audio file (or) in which the motion end timing of the motion ID associated with the end alignment flag (that is, the end timing of the motion expression, which corresponds to the motion end timing described later) is associated with this motion ID. It is a flag for specifying whether or not to match the end timing of the synthetic voice) (that is, the end timing of the voice expression, which corresponds to the voice end timing described later). Therefore, for the motion in which the end alignment flag is set, the body controller 360 creates a schedule for operating the voice module 340 and the motion module 350 so that the end timing of the audio output and the end timing of the motion match. .. For motions for which the end alignment flag is not set, the body controller 360 schedules the voice module 340 and the motion module 350 to operate, for example, so that the start timing of the audio output and the start timing of the motion match. You may create it.
  • the path to the motion file may be a file path indicating the storage position of the actual motion file (CVS file, TXT file, etc.) in the file storage area 303.
  • the file storage area 303 in FIG. 39 may be a storage area for storing one or more motion files (and audio files) that can be executed by the autonomous mobile body 10.
  • the file storage area 303 may be arranged in the autonomous mobile body 10 or in the information processing server 20.
  • the body controller 360 analyzes the motion file acquired from the file storage area 303 when scheduling the voice expression and the motion expression, so that the actual motion time of each motion, the meaning of each motion (category, etc.), etc.
  • Information (hereinafter referred to as motion structure information) can be acquired in advance.
  • the body controller 360 determines the actual utterance time for each phrase after disassembling the text into morphemes, the actual utterance time for each mora, the actual utterance time for the entire text, and the meaning of each morpheme based on the voice file or the synthesized voice.
  • Information such as (category, etc.) (hereinafter referred to as voice structure information) can be acquired in advance.
  • the body controller 360 may acquire voice structure information by analyzing the voice file (or synthetic voice) by itself, or inputs the voice file (or synthetic voice) to the voice module 340.
  • the voice structure information may be acquired from the voice module 340, or the voice structure information is associated with the voice file or text in advance, and the voice structure information is acquired when the voice file or text is acquired. You may.
  • the body controller 360 schedules the timing of starting or ending the reproduction of the voice file (or synthetic voice) and the timing of starting or ending the motion based on the motion structure information and the voice structure information acquired in advance as described above. By doing so, the voice expression and the motion expression are coordinated.
  • the body controller 360 may acquire the motion structure information by analyzing the motion file by itself, or may input the motion file into the motion module 350 and acquire the motion structure information from the motion module 350.
  • Motion structure information is associated with the motion file in advance, and the motion structure information may be acquired at the same time as the motion file is acquired.
  • FIG. 42 is a flowchart showing an example of the cooperative operation according to the present embodiment.
  • FIG. 42 and the following description for simplification, the case where the phrase of the text notified from the information processing server 20 to the autonomous mobile body 10 is not considered is illustrated. Further, in the following description, for the sake of clarification, the operation of the body controller 360 will be focused on.
  • the body controller 360 receives the text and the intention to be spoken from the information processing server 20 via the network 40 and the communication unit 170 (step S101). ..
  • the text and intent may be transmitted to the autonomous mobile 10 in either a push-type or pull-type manner.
  • the body controller 360 searches the combination list 301 to determine whether or not the received text is registered in the combination list 301 (step S102).
  • the body controller 360 When the text is not registered in the combination list 301 (NO in step S102), the body controller 360 generates a synthetic voice of the text (step S103) and acquires the actual utterance time of the generated synthetic voice (step S104). ), Proceed to step S113.
  • the actual utterance time of the synthetic voice may be acquired by various methods such as inputting the data of the synthetic voice into the voice module 340 and notifying the body controller 360 from the voice module 340.
  • the body controller 360 determines whether or not the voice data is associated with the text in the combination list 301 (step S105).
  • step S105 When the voice data is not associated with the text in the combination list 301 (NO in step S105), the body controller 360 generates a synthetic voice of the text (step S106), and proceeds to step S108.
  • the body controller 360 acquires this audio file (step S107) and proceeds to step S108.
  • step S108 the body controller 360 acquires the actual utterance time of the generated synthetic voice or the acquired voice file (step S108) in the same manner as in step S104, and proceeds to step S109.
  • the actual utterance time of the voice file may be acquired by various methods, for example, the voice file is input to the voice module 340 and the voice module 340 notifies the body controller 360, as in the case of the synthetic voice.
  • step S109 the body controller 360 determines whether or not the motion ID or the category ID is associated with the text in the combination list 301.
  • step S109 When the motion ID or category ID is not associated with the text in the combination list 301 (NO in step S109), the body controller 360 proceeds to step S113.
  • the body controller 360 determines whether or not the category ID is associated with the text (step S110). ).
  • the body controller 360 selects one or more motion IDs from the categories specified by the category ID (step S111), and steps. Proceed to S114.
  • the body controller 360 transfers the motion file to the motion file in the combination list 301. Based on the path, the motion file specified by the motion ID is acquired from the file storage area 303 (step S112), and the process proceeds to step S115.
  • step S113 the body controller 360 selects one of the motion IDs from the motions belonging to the category corresponding to the intention received from the information processing server 20, and proceeds to step S114.
  • step S114 the body controller 360 acquires the motion file specified by the motion ID selected from the file storage area 303 based on the path to the motion file in the combination list 301, and proceeds to step S115.
  • step S115 the body controller 360 acquires the actual operation time of the motion by analyzing the motion file acquired in step S112 or S114.
  • the body controller 360 schedules the voice expression by voice output and the motion expression by motion based on the acquired voice file or synthetic voice and motion file, and the acquired actual speech time and actual operation time (step). S116).
  • the body controller 360 inputs the acquired voice file or synthetic voice and the generated utterance schedule to the voice module 340, and also inputs the acquired motion file and the generated operation schedule to the motion module 350 ( Step S117).
  • the voice module 340 and the motion module 350 execute the voice file (or synthetic voice) and the motion file according to the schedule, respectively.
  • the autonomous mobile body 10 executes voice expression and motion expression coordinated with each other.
  • the body controller 360 determines whether or not to end this operation (step S118), and if it ends (YES in step S118), ends this operation. On the other hand, if it does not end, the body controller 360 returns to step S101 and waits for the next text and intention to be received from the information processing server 20.
  • FIG. 43 is a sequence diagram for explaining a cooperative operation according to an example of the first pattern of the present embodiment.
  • the horizontal axis is the time axis.
  • the motion (motion) and the voice reproduction (voice) are started at the same timing, and the unit motion is 4 during the period during which the main voice “good morning” is being played. It is looped once, and the playback of the main audio is finished during the fifth loop.
  • the end of voice reproduction is notified from the voice module 340 to the body controller 360 at the timing when the reproduction of the main voice is completed.
  • the body controller 360 receives the end notification from the voice module 340
  • the body controller 360 inputs the end instruction of the loop of the unit motion to the motion module 350.
  • the motion module 350 receives the end instruction from the body controller 360
  • the motion module 350 ends the execution of the main motion at the end of the unit motion currently being executed.
  • FIG. 44 is a sequence diagram for explaining a cooperative operation according to an example of the second pattern of the present embodiment.
  • the horizontal axis is the time axis.
  • the motion (motion) and the voice reproduction (voice) are started at different timings, and the unit motion is looped four times until the playback of the main voice “surprise” is completed. Then, the playback of the main audio is finished during the fifth loop. That is, in the second pattern example, it is not necessary that the motion (motion) and the voice reproduction (voice) are started at the same timing.
  • the loop end instruction is input to the motion module 350 at the timing when the reproduction of the main voice is completed, and the unit motion currently being executed is input. Finally, the loop execution of the unit motion ends.
  • FIG. 45 is a sequence diagram for explaining the cooperative operation according to the third pattern example of the present embodiment.
  • the horizontal axis is the time axis.
  • one text is composed of three phrases.
  • the main motion and the main voice may operate in cooperation with each clause.
  • the cooperative operation of each clause may be the cooperative operation described in the first or second pattern example, and for example, various other cooperative operations such as matching the operation end timing and the voice end timing. It may be a cooperative operation of.
  • the body controller 360 may create an utterance schedule and an operation schedule for each phrase. Then, the schedule created for each clause may be input to each of the voice module 340 and the motion module 350 from the body controller 360, or all the schedules may be input at once.
  • FIG. 46 is a sequence diagram for explaining a cooperative operation according to an example of the fourth pattern of the present embodiment.
  • the horizontal axis is the time axis.
  • one text is composed of two phrases.
  • the second and subsequent clauses are next without waiting for the end of the last unit motion in the continuous motion executed for the previous clause. Playback of the main voice for a phrase may be started, and then the main motion for the next phrase may be scheduled to start as soon as the execution of the last unit motion in the previous phrase is completed.
  • FIG. 47 is a sequence diagram for explaining a cooperative operation according to the fifth pattern example of the present embodiment.
  • the horizontal axis is the time axis.
  • a specific motion hereinafter referred to as a fixed motion
  • the body controller 360 may schedule to start the main motion as soon as the execution of the fixed motion is completed. ..
  • FIG. 48 is a sequence diagram for explaining the cooperative operation according to the sixth pattern example of the present embodiment.
  • the horizontal axis is the time axis.
  • the body controller 360 is based on the actual operation time of the fixed motion and the actual utterance time of the main voice.
  • the timing for ending the motion may be back-calculated, and an end instruction for ending the main motion at the back-calculated timing may be given to the motion module 350.
  • FIG. 49 is a sequence diagram for explaining the cooperative operation according to the 7th pattern example of the present embodiment.
  • the horizontal axis is the time axis.
  • the main motion may be a continuous motion composed of different unit motions A to C.
  • the unit motions A to C may be, for example, motions selected in advance from motions belonging to the same or different (including similar) categories, or motions selected in advance according to a predetermined rule such as round robin or randomly selected.
  • An example of the first pattern which is a basic pattern, is, for example, when the timing control is not specified in the combination list 301 for the text notified from the information processing server 20 (for example, when the end alignment flag is not set). It may be executed when the notified text is not registered in the combination list 301. If the text notified from the information processing server 20 is not registered in the combination list 301, the body controller 360 is notified of the entire motion file registered in the file storage area 303 or from the information processing server 20. You may select a motion file from a category that suits your intention, according to a predetermined rule such as round robin, or at random.
  • the second pattern example may be executed, for example, when the voice start timing is specified.
  • Various methods are adopted for specifying the voice start timing, for example, registering it in the combination list 301, embedding it in a motion file, notifying it from the information processing server 20, or specifying it by the body controller 360. be able to.
  • the third pattern example may be executed, for example, when the text notified from the information processing server 20 is composed of a plurality of clauses and each clause is registered as text in the combination list 301. For example, when the information processing server 20 notifies a text consisting of three phrases "Hey, I, (name)" and each phrase is separately registered in the combination list 301, each phrase is registered. When different motions are associated with (text), it is necessary to coordinate the voice output and the motion in each clause, so that the third pattern example can be used.
  • the fourth pattern example may be executed, for example, when the text notified from the information processing server 20 is composed of a plurality of clauses and a part of the plurality of clauses is not registered in the combination list 301.
  • the information processing server 20 notifies the text consisting of two phrases "Hey, who are you?", And although "Hey” is registered in the combination list 301, "Who are you?" Is the combination list 301. If it is not melted, for example, "Hey Hey” is executed in the first pattern example, "Who are you?" Is output as synthetic voice, and a predetermined rule or a randomly selected unit motion is output. It can be repeated as many times as necessary.
  • the fifth pattern example is executed, for example, when the text notified from the information processing server 20 is registered in the combination list 301, and the combination list 301 specifies that the fixed motion is to be executed first. Good.
  • the motion looped after the fixed motion may be the motion specified in the combination list 301, or may be a motion selected according to a predetermined rule or randomly.
  • the sixth pattern example is executed, for example, when the text notified from the information processing server 20 is registered in the combination list 301, and the combination list 301 specifies that the fixed motion is to be executed last. Good.
  • the motion looped before the fixed motion may be the motion specified in the combination list 301, or may be a motion selected according to a predetermined rule or randomly. Further, there may be a stop period between the previously executed main motion and the last fixed motion in order to match the voice end timing and the operation end timing (that is, for time adjustment).
  • the seventh pattern example may be executed, for example, when the text notified from the information processing server 20 is registered in the combination list 301 and only the motion category is specified in the combination list 301.
  • the interrupt processing according to the present embodiment may be interrupted by both the voice expression and the motion expression, or may be interrupted by one of the voice expression and the motion expression.
  • the case where both the voice expression and the motion expression are interrupted is illustrated in the first example and the second example, and the case where only the motion expression is interrupted in the third example and the fourth example is illustrated.
  • the case where only the voice expression is interrupted will be illustrated.
  • FIG. 50 is a diagram showing a case where interrupt processing occurs during the cooperative operation according to the first example. Note that FIG. 50 illustrates a case where interrupt processing occurs while the autonomous mobile body 10 is watching television.
  • the autonomous mobile body 10 executes a motion such as staring at the television or nodding to the voice from the television as its motion expression, and as a voice expression. , Outputs voice that cooperates with motions such as "I see” and "How” (step S11).
  • step S11 the user enters the field of view of the autonomous mobile body 10, or enters the same space as the autonomous mobile body 10 (for example, a room).
  • the autonomous mobile body 10 executes interrupt processing for each of the voice expression and the motion expression in step S11.
  • the autonomous mobile body 10 executes a motion such as a surprise for discovering the user or following the user as its motion expression, and cooperates with the surprise as a voice expression.
  • Outputs voice that cooperates with motion such as "Let's play" that cooperates with tracking. That is, the autonomous mobile body 10 also executes the cooperative operation in the interrupt processing in step S12.
  • the autonomous mobile body 10 executes interrupt processing for each of the voice expression and the motion expression in steps S11 or S12.
  • the autonomous moving body 10 has, for example, blinking the LED of the eye portion 510 corresponding to the eyes (blinking of the eyes), following the user by rotating the substrate 505 corresponding to the head (following with the neck), and the like.
  • a voice expression a voice that cooperates with a motion such as "Wow” or "SE (sound according to acceleration)" is output. That is, the autonomous mobile body 10 also executes the cooperative operation in the interrupt processing in step S13.
  • FIG. 51 is a diagram showing a case where interrupt processing occurs during the cooperative operation according to the second example. Note that FIG. 51 illustrates a case where the autonomous moving body 10 finds a cliff (edge of the top plate) in front of the traveling direction while traveling on the top plate of the table.
  • the autonomous mobile body 10 receives from the outside (for example, the information processing server 20) or plans an action plan by itself as its operation expression, for example. According to this, the motion of traveling on the top plate is executed, and as a voice expression, a voice that cooperates with the motion such as "Bun" is output (step S21).
  • the autonomous mobile body 10 interrupts each of the voice expression and the operation expression in step S21. Execute the process. In this interrupt process, the autonomous mobile body 10 executes a motion such as an action of avoiding the edge of the top plate or an action of stopping as its motion expression, and as a voice expression, "Abunai” or "SE (brake sound)". ) ”And other motion-coordinated sounds are output. That is, the autonomous mobile body 10 also executes the cooperative operation in the interrupt processing in step S22.
  • FIG. 52 is a diagram showing a case where interrupt processing occurs during the cooperative operation according to the third example. Note that FIG. 52 illustrates a case where the autonomous mobile body 10 is grasped and lifted by the user while reading important information such as news, a weather forecast, and a message notified by SNS.
  • the autonomous mobile body 10 executes, for example, a motion facing the user as its motion expression, and expresses it by voice. As a result, a voice such as a weather forecast "Today's weather is sunny" is output (step S31).
  • step S31 During the reading aloud in step S31, for example, when a situation such as gripping or lifting occurs by the user as in step S32, the autonomous mobile body 10 executes an interrupt process for the operation expression in step S31.
  • the autonomous mobile body 10 blinks the LED of the eye portion 510 corresponding to the eye (blinking of the eye) and follows the user by rotating the substrate 505 corresponding to the head (as its operation expression). Perform motions such as (follow with the neck).
  • the autonomous mobile body 10 expresses that it is aware of the occurrence of processing to be interrupted.
  • the interrupt for the voice expression does not occur. Therefore, the autonomous mobile body 10 continues to read out important information such as the weather forecast as a voice expression.
  • FIG. 53 is a diagram showing a case where interrupt processing occurs during the cooperative operation according to the fourth example. Note that FIG. 53 illustrates a case where the autonomous mobile body 10 discovers another user during a conversation with the user.
  • the autonomous mobile body 10 executes, for example, an action (motion) coordinated with the content of the conversation as its motion expression, and with the user as a voice expression. (Step S41).
  • the autonomous mobile body 10 executes an interrupt process for the operation expression in step S41. To do.
  • the autonomous mobile body 10 executes a motion such as following another user (following another user with a face) by rotating the substrate 505 corresponding to the head, for example, as its motion expression.
  • the autonomous mobile body 10 expresses that it is aware of the occurrence of processing to be interrupted.
  • the interrupt for the voice expression does not occur. Therefore, the autonomous mobile body 10 continues the conversation with the user as a voice expression.
  • FIG. 54 is a diagram showing a case where interrupt processing occurs during the cooperative operation according to the fifth example. Note that FIG. 54 illustrates a case where the autonomous mobile body 10 is struck by the user during an avoidance operation such as a cliff or an obstacle.
  • the autonomous mobile body 10 executes an avoidance operation (motion) for avoiding a cliff, an obstacle, or the like as its motion expression, and as a voice expression, " A voice output such as "Abunai” is executed (step S51).
  • step S51 for example, when a process to be interrupted occurs, such as being struck by the user as in step S52, the autonomous mobile body 10 performs an interrupt process for the voice expression in step S51. Execute. In this interrupt process, the autonomous mobile body 10 outputs, for example, "what" or the like as its voice expression. As a result, the autonomous mobile body 10 expresses that it is aware of the occurrence of processing to be interrupted. However, in the fifth example, the interrupt for the operation expression does not occur. Therefore, the autonomous mobile body 10 continues the avoidance motion as an motion expression.
  • FIG. 55 is a diagram showing a case where interrupt processing occurs during the cooperative operation according to the sixth example. Note that FIG. 55 illustrates a case where a user appears during an important operation such as when the autonomous mobile body 10 is moving to the charger.
  • the autonomous mobile body 10 may, for example, move to the charger as its operation expression.
  • the important motion (motion) of the above is executed, and the voice output such as "I'm hungry” is executed as the voice expression (step S61).
  • step S61 the user enters the field of view of the autonomous mobile body 10 or enters the same space as the autonomous mobile body 10 (for example, a room).
  • the autonomous mobile body 10 executes an interrupt process for the voice expression in step S61.
  • the autonomous mobile body 10 outputs a voice such as a greeting such as "Good morning" as its voice expression, for example.
  • the autonomous mobile body 10 expresses that it is aware of the occurrence of processing to be interrupted.
  • the interrupt for the operation expression does not occur. Therefore, the autonomous mobile body 10 continues an important operation as an operation expression.
  • 1.18.7 Representation when an interrupt occurs As in the above example, when an interrupt process occurs, the autonomous mobile unit 10 may execute an operation or voice output to express that fact. Further, this action may be a cooperative action of the voice expression and the action expression.
  • a specific SE, a fixed word, or the like may be output as voice in cooperation with the motion to be caused.
  • the motion registered in the motion DB 302 and the motion file stored in the file storage area 303 may be added as appropriate.
  • the information about each item (motion ID, category ID, emotion, loop enable / disable flag, end matching flag) to be registered in the motion DB 302 shown in FIG. 41 and the motion file are, for example.
  • the information processing server 20 or another server connected to the network 40 may be downloaded to the autonomous mobile body 10.
  • Information about each item downloaded to the autonomous mobile body 10 for one motion is stored in one record of the motion DB 302. Further, a file path (path to the motion file) indicating the storage position in the file storage area 303 of the motion file, which is the substance of the motion, is also registered in this record. As a result, a new motion is registered in the autonomous mobile body 10.
  • the newly added motion may be managed in the motion DB 302 so as to belong to a category independent of other motions, based on, for example, the vendor that provides the motion, the meaning and content of the motion, and the like.
  • the autonomous mobile body 10 can be customized by each user, for example, by causing the autonomous mobile body 10 to preferentially execute a motion provided by a specific vendor.
  • the body controller 360 has a breath sound longer than usual in the start motion, for example, when expressing a text having more characters than a certain threshold by voice expression and motion expression.
  • the utterance schedule may be scheduled so as to output a strong breath sound or the like (for example, a long breath sound "ha").
  • the body controller 360 for example, when expressing a text having a number of characters less than a certain threshold by voice expression and motion expression, a breath sound shorter than usual, a weak breath sound, etc. in the start motion (for example, "ha").
  • the utterance schedule may be scheduled to output a short breath sound).
  • the strength of the breath sound is determined by, for example, "emphasis” or “prosody” in synthetic speech data described by Speech Synthesis Markup Language (SSML). It may be realized by using a specific tag such as ".
  • the breath sound may be, for example, an audio file recording what is emitted by a person or an animal.
  • the breath sound recorded together with the recording of the audio file may be used. In that case, by cutting out the breath sound at the beginning of the audio file of the text (text), it is possible to create an audio file of the breath sound for this text (text).
  • the same breath sound may be used for texts with the same first phrase.
  • the body controller 360 may analyze the text to identify the first phrase, and schedule the audio file of the breath sound preset for this phrase to be played back in the start motion.
  • the breath sound includes the breath sound after the utterance of the text (shortness of breath, etc.) and the breath sound during the utterance of the text (breathing, etc.). Also exists. Therefore, for example, when the body controller 360 expresses a text having more characters than a certain threshold by voice expression and motion expression, it expresses a breath sound or the like (“hahaa”, “long time”, etc.) indicating shortness of breath or the like in the end motion. The utterance schedule may be scheduled as such.
  • the body controller 360 when the body controller 360 expresses a text having a number of characters larger than a certain threshold by voice expression and motion expression, the body controller 360 expresses a breath sound or the like (“su” or the like) indicating breathing or the like in the middle of the main motion.
  • the utterance schedule may be scheduled.
  • the body controller 360 uses the situation around the autonomous mobile body 10 (for example, the baby is sleeping in the same room or the current time is midnight, etc.).
  • the voice quality (sound quality) of the output voice may be changed depending on the internal situation of the autonomous mobile body 10 (for example, the situation where the remaining battery level is low or the acceleration is large).
  • the voice quality may be raised or lowered according to the acceleration detected by the inertial sensor 525. Further, when the remaining battery level is low during the speech of the autonomous mobile body 10, the voice quality (sound quality) may be changed according to the remaining battery level.
  • voice quality can also be realized by using a specific tag of SSML, as with the breath sound described above.
  • the autonomous mobile body 10 may intentionally bite or make a mistake during speech. As a result, the character of the autonomous mobile body 10 can be enhanced, so that the user can have a feeling of familiarity.
  • FIG. 56 is a block diagram showing a hardware configuration example of the information processing server 20 according to the first embodiment of the present disclosure.
  • the information processing server 20 includes, for example, a processor 871, a ROM 872, a RAM 873, a host bus 874, a bridge 875, an external bus 876, an interface 877, an input device 878, and an output device 879. It has a storage 880, a drive 881, a connection port 882, and a communication device 883.
  • the hardware configuration shown here is an example, and some of the components may be omitted. Further, components other than the components shown here may be further included.
  • the processor 871 functions as, for example, an arithmetic processing unit or a control device, and controls all or a part of the operation of each component based on various programs recorded in the ROM 872, the RAM 873, the storage 880, or the removable recording medium 901. ..
  • the ROM 872 is a means for storing a program read into the processor 871 and data used for calculation.
  • the RAM 873 temporarily or permanently stores, for example, a program read into the processor 871 and various parameters that change as appropriate when the program is executed.
  • the processors 871, ROM 872, and RAM 873 are connected to each other via, for example, a host bus 874 capable of high-speed data transmission.
  • the host bus 874 is connected to the external bus 876, which has a relatively low data transmission speed, via, for example, the bridge 875.
  • the external bus 876 is connected to various components via the interface 877.
  • Input device 8708 For the input device 878, for example, a mouse, a keyboard, a touch panel, buttons, switches, levers, and the like are used. Further, as the input device 878, a remote controller (hereinafter referred to as a remote controller) capable of transmitting a control signal using infrared rays or other radio waves may be used. Further, the input device 878 includes a voice input device such as a microphone.
  • the output device 879 provides the user with acquired information such as a display device such as a CRT (Cathode Ray Tube), an LCD, or an organic EL, an audio output device such as a speaker or headphones, a printer, a mobile phone, or a facsimile. It is a device that can notify visually or audibly. Further, the output device 879 according to the present disclosure includes various vibration devices capable of outputting a tactile stimulus.
  • the storage 880 is a device for storing various types of data.
  • a magnetic storage device such as a hard disk drive (HDD), a semiconductor storage device, an optical storage device, a magneto-optical storage device, or the like is used.
  • the drive 881 is a device that reads information recorded on a removable recording medium 901 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, or writes information to the removable recording medium 901.
  • a removable recording medium 901 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory
  • the removable recording medium 901 is, for example, a DVD media, a Blu-ray (registered trademark) media, an HD DVD media, various semiconductor storage media, and the like.
  • the removable recording medium 901 may be, for example, an IC card equipped with a non-contact type IC chip, an electronic device, or the like.
  • connection port 882 is a port for connecting an external connection device 902 such as a USB (Universal Serial Bus) port, an IEEE1394 port, a SCSI (Small Computer System Interface), an RS-232C port, or an optical audio terminal. is there.
  • an external connection device 902 such as a USB (Universal Serial Bus) port, an IEEE1394 port, a SCSI (Small Computer System Interface), an RS-232C port, or an optical audio terminal. is there.
  • the externally connected device 902 is, for example, a printer, a portable music player, a digital camera, a digital video camera, an IC recorder, or the like.
  • the communication device 883 is a communication device for connecting to a network, and is, for example, a communication card for wired or wireless LAN, Bluetooth (registered trademark), or WUSB (Wireless USB), a router for optical communication, and ADSL (Asymmetric Digital). A router for Subscriber Line), a modem for various communications, and the like.
  • FIG. 57 is a front view of the autonomous mobile body 10 according to the second embodiment of the present disclosure
  • FIG. 58 is a side view of the autonomous mobile body 10 according to the second embodiment of the present disclosure.
  • the case where the main body (body) of the autonomous mobile body 10 is an oblong body has been exemplified. Therefore, for example, when the autonomous moving body 10 falls on an inclined surface or the like, there is a possibility that the autonomous moving body 10 rolls along the inclined surface as it is, falls from a table or the like, or collides with a wall or the like.
  • a convex portion 720 is provided on a part of the side surface of the autonomous moving body 10.
  • the autonomous moving body 10 falls on an inclined surface or the like, it is possible to suppress the rolling of the autonomous moving body 10, so that the autonomous moving body 10 may fall from a table or the like or collide with a wall or the like. It becomes possible to suppress.
  • interrupt processing occurs during communication with the user, it is possible to execute interrupt processing in a more natural way while reducing the discomfort given to the user.
  • the present technology can also have the following configurations.
  • a voice module that outputs sound or voice according to the input action plan
  • a motion module that executes actions according to the input action plan
  • a body controller that creates an action plan for each of the voice module and the motion module, With The body controller Acquires the voice data to be output as voice and the motion data to execute the operation, Based on the voice data and the motion data, a first action plan of the voice module and a second action plan of the motion module are created.
  • An information processing device that inputs the first action plan to the voice module and inputs the second action plan to the motion module.
  • the body controller creates the first and second action plans based on the first time information for each phrase of the voice data and the second time information of the motion data (2) to (2).
  • the information processing apparatus according to any one of 5).
  • the body controller acquires the motion data different for each clause, and based on the first and second time information, the output start timing or output end timing of the voice data and the execution of the motion data are executed for each clause.
  • the body controller acquires the motion data different for each phrase, and when the audio output of the previous phrase is completed, the body controller outputs the audio of the next phrase without waiting for the execution of the motion data for the previous phrase to end.
  • the first and second action plans are created so as to start and execute the motion data for the next clause as soon as the execution of the motion data for the previous clause is completed.
  • the information processing apparatus according to 7).
  • the body controller creates the second action plan so as to execute different motion data until the output of the voice data is completed.
  • the information processing device according to (2). (11) It also has an acquisition unit that acquires text from an external device.
  • the information processing device according to any one of (1) to (10), wherein the body controller acquires the voice data and the motion data based on the text.
  • the information processing device (12) The information processing device according to (11), wherein the acquisition unit acquires the text from the external device via a predetermined network. (13) The acquisition unit further acquires attribute information related to the text from the external device, and the body controller acquires the voice data and the motion data based on the text and the attribute information. The information processing apparatus according to 12). (14) A combination list that holds the correspondence between the text and the voice data and the motion data is further provided. The information processing device according to any one of (11) to (13), wherein the body controller acquires the voice data and the motion data by referring to the combination list based on the text. (15) A storage unit that stores the motion data and A motion database that associates the first identification information for identifying the motion data with the position information indicating the position of the motion data in the storage unit.
  • the motion database manages the motion data by category and manages the motion data.
  • the combination list associates the text with the second identification information for identifying the category.
  • the body controller has one or more first identification information belonging to the category specified by the second identification information in the motion database based on the second identification information specified from the combination list based on the text.
  • the information processing apparatus wherein one of the first identification information is selected from the above, and the motion data is acquired from the storage unit based on the selected first identification information.
  • the body controller When an interrupt process occurs while the voice module and the motion module are executing the first action plan and the second action plan, the body controller performs the interrupt process on either the voice module or the motion module.
  • the information processing apparatus according to any one of (1) to (16) above.
  • the body controller executes the interrupt process on both the voice module and the motion module.
  • the information processing apparatus according to any one of (1) to (16) above.
  • An information processing method executed in a device including a voice module that outputs a sound or voice according to an input action plan and a motion module that executes an operation according to the input action plan.
  • a program for operating a processor mounted on a device including a voice module that outputs sound or voice according to an input action plan and a motion module that executes an operation according to the input action plan.
  • the process of acquiring the audio data to be output as audio and the motion data to execute the operation A process of creating a first action plan of the voice module and a second action plan of the motion module based on the voice data and the motion data.

Landscapes

  • Engineering & Computer Science (AREA)
  • Computational Linguistics (AREA)
  • Health & Medical Sciences (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • Acoustics & Sound (AREA)
  • Multimedia (AREA)
  • Toys (AREA)
PCT/JP2020/042990 2019-12-10 2020-11-18 情報処理装置、その制御方法及びプログラム Ceased WO2021117441A1 (ja)

Priority Applications (2)

Application Number Priority Date Filing Date Title
US17/781,275 US20220410023A1 (en) 2019-12-10 2020-11-18 Information processing device, control method of the same, and program
JP2021563818A JPWO2021117441A1 (https=) 2019-12-10 2020-11-18

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2019-222604 2019-12-10
JP2019222604 2019-12-10

Publications (1)

Publication Number Publication Date
WO2021117441A1 true WO2021117441A1 (ja) 2021-06-17

Family

ID=76329811

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2020/042990 Ceased WO2021117441A1 (ja) 2019-12-10 2020-11-18 情報処理装置、その制御方法及びプログラム

Country Status (3)

Country Link
US (1) US20220410023A1 (https=)
JP (1) JPWO2021117441A1 (https=)
WO (1) WO2021117441A1 (https=)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2024071166A1 (ja) * 2022-09-29 2024-04-04 日東電工株式会社 ロボット
WO2024203004A1 (ja) * 2023-03-29 2024-10-03 ソニーグループ株式会社 自律移動体及び動作制御方法

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
USD1043861S1 (en) * 2024-01-11 2024-09-24 Zhengxin Li Plush toy

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2004283959A (ja) * 2003-03-20 2004-10-14 Sony Corp ロボット装置、その動作制御方法、及びプログラム
JP2011141305A (ja) * 2010-01-05 2011-07-21 Nec Corp 音声合成装置、ロボット装置、音声合成方法、及び、プログラム
JP2015013351A (ja) * 2013-07-08 2015-01-22 有限会社アイドリーマ ロボットを制御するためのプログラム
WO2017199565A1 (ja) * 2016-05-20 2017-11-23 シャープ株式会社 ロボット、ロボットの動作方法、及びプログラム
JP2017213612A (ja) * 2016-05-30 2017-12-07 トヨタ自動車株式会社 ロボットおよびロボットの制御方法
JP2018185362A (ja) * 2017-04-24 2018-11-22 富士ソフト株式会社 ロボットおよびその制御方法
WO2019160090A1 (ja) * 2018-02-16 2019-08-22 日本電信電話株式会社 非言語情報生成装置、方法、及びプログラム
WO2019163279A1 (ja) * 2018-02-26 2019-08-29 ソニー株式会社 情報処理装置、情報処理方法、およびプログラム

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR100925707B1 (ko) * 2009-07-17 2009-11-10 주식회사 로보메이션 자동 공연이 가능한 로봇용 공연장치
KR20140092136A (ko) * 2013-01-15 2014-07-23 주식회사 이드웨어 언어학습 어젬블리

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2004283959A (ja) * 2003-03-20 2004-10-14 Sony Corp ロボット装置、その動作制御方法、及びプログラム
JP2011141305A (ja) * 2010-01-05 2011-07-21 Nec Corp 音声合成装置、ロボット装置、音声合成方法、及び、プログラム
JP2015013351A (ja) * 2013-07-08 2015-01-22 有限会社アイドリーマ ロボットを制御するためのプログラム
WO2017199565A1 (ja) * 2016-05-20 2017-11-23 シャープ株式会社 ロボット、ロボットの動作方法、及びプログラム
JP2017213612A (ja) * 2016-05-30 2017-12-07 トヨタ自動車株式会社 ロボットおよびロボットの制御方法
JP2018185362A (ja) * 2017-04-24 2018-11-22 富士ソフト株式会社 ロボットおよびその制御方法
WO2019160090A1 (ja) * 2018-02-16 2019-08-22 日本電信電話株式会社 非言語情報生成装置、方法、及びプログラム
WO2019163279A1 (ja) * 2018-02-26 2019-08-29 ソニー株式会社 情報処理装置、情報処理方法、およびプログラム

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2024071166A1 (ja) * 2022-09-29 2024-04-04 日東電工株式会社 ロボット
WO2024203004A1 (ja) * 2023-03-29 2024-10-03 ソニーグループ株式会社 自律移動体及び動作制御方法

Also Published As

Publication number Publication date
JPWO2021117441A1 (https=) 2021-06-17
US20220410023A1 (en) 2022-12-29

Similar Documents

Publication Publication Date Title
JP7626179B2 (ja) 情報処理装置、及び、情報処理方法
JP7502520B2 (ja) ロボット、ロボットの制御方法及びプログラム
JP7439826B2 (ja) 情報処理装置、情報処理方法、及びプログラム
WO2021117441A1 (ja) 情報処理装置、その制御方法及びプログラム
US12197188B2 (en) Information processing apparatus and information processing method
US12311539B2 (en) Information processing apparatus and control method
US12001226B2 (en) Information processing apparatus, control method, and program
US20250244749A1 (en) Information processing apparatus and information processing method
WO2023037608A1 (ja) 自律移動体、情報処理方法、及び、プログラム
WO2020213244A1 (ja) 情報処理装置、情報処理方法、及びプログラム
JP7733695B2 (ja) 行動制御システム
JP2024155870A (ja) 行動制御システム
JP2024164821A (ja) 行動制御システム
JP2025031623A (ja) 制御システム
JP2025029592A (ja) 制御システム
JP2025026426A (ja) 制御システム
JP2024159728A (ja) 電子機器
JP2025022825A (ja) 行動制御システム
JP2025013314A (ja) 制御システム
JP2024155799A (ja) 電子機器
JP2025036198A (ja) 制御システム
JP2025013313A (ja) 制御システム
JP2025001593A (ja) 情報処理システム

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20898708

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2021563818

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20898708

Country of ref document: EP

Kind code of ref document: A1