WO2020022371A1

WO2020022371A1 - Robot, method for controlling robot, and control program

Info

Publication number: WO2020022371A1
Application number: PCT/JP2019/028975
Authority: WO
Inventors: 要林; 秀哉南地; 泰士深谷; 直人吉岡
Original assignee: ＧｒｏｏｖｅＸ株式会社
Priority date: 2018-07-26
Filing date: 2019-07-24
Publication date: 2020-01-30
Also published as: JPWO2020022371A1

Abstract

A robot comprising: an execution information acquisition unit that acquires execution information for executing a notification operation to notify a person to be notified of information to be executed; a search unit that searches for the person to be notified on the basis of the execution information acquired by the execution information acquisition unit; and a notification operation execution unit that executes the notification operation, for the person to be notified searched for by the search unit, on the basis of the execution information.

Description

Robot and its control method and control program

The present invention relates to a robot, a control method thereof, and a control program.

Conventionally, there is a robot that takes an image with a camera while moving autonomously in a house, recognizes an indoor space from the captured image, sets a movement route based on the recognized space, and moves indoors. The setting of the robot movement route is performed by the user creating a map in advance that defines the route on which the robot moves. The robot can move on a route determined based on the created map (for example, see Patent Literature 1).

There is also a robot that can output an alarm at a predetermined time (for example, see Patent Document 2).

JP 2016-103277 A JP-A-2017-119320

However, the conventional robot may not be able to notify the target person of information when a specific target to be notified of information such as an alarm is not present near the robot.

The present invention has been made in view of the above circumstances, and in one embodiment, provides a robot capable of notifying a specific notification target person of information and a control method and a control program thereof. One purpose.

(1) In order to solve the above-described problem, the robot according to the embodiment has an execution information acquisition unit that acquires execution information for executing an operation of informing information to be notified to a notification target person, and an execution information acquisition unit. And a notification operation executing unit that executes a notification operation based on the execution information for the notification target person searched for by the searching unit based on the execution information acquired in step (1).

{Circle around (2)} In the robot of the embodiment, the execution information acquisition unit acquires location information relating to a location designated by the user as execution information.

(3) In the robot of the embodiment, the execution information obtaining unit obtains, from the user terminal, location information specified by the user operating the map displayed on the user terminal operated by the user. .

(4) In the robot according to the embodiment, the search unit further searches for a notification target person based on a captured image of the surrounding space.

(5) In the robot of the embodiment, the search unit searches for a notification target person by recognizing a person included in the captured image.

(6) The robot according to the embodiment further includes a movement control unit that controls the movement mechanism, wherein the search unit calculates a movement path of the movement mechanism based on the execution information, and the movement control unit calculates the movement path. The moving mechanism is controlled based on the moving path.

{Circle around (7)} In the robot of the embodiment, the search unit calculates the movement route further based on the restriction information for restricting the movement by the moving mechanism.

(8) In the robot of the embodiment, the robot further includes a marker recognizing unit that recognizes a predetermined marker included in a captured image of the surrounding space, and the movement control unit is configured to perform the operation based on the marker recognized by the marker recognizing unit. Control the moving mechanism.

(9) Further, in the robot of the embodiment, the robot further includes a state information acquisition unit that acquires state information relating to the state of the notification target person searched by the search unit, and the notification operation execution unit is acquired by the state information acquisition unit. The notification operation is changed according to the state of the notification.

(10) In the robot of the embodiment, the state information obtaining unit obtains a state of whether the notification target person is sleeping or waking up, and the notification operation execution unit determines the state as the notification operation. When the user is asleep, an awakening operation for waking up the notification target person is executed, and when the state is awakening, a greeting operation for the notification target person is executed.

(11) In the robot of the embodiment, the execution information acquisition unit acquires execution information associated with the notification target person, and the notification operation execution unit executes the notification operation associated with the searched notification target person. I do.

(12) In the robot of the embodiment, the execution information acquisition unit acquires, as execution information, intimacy information indicating the level of intimacy between the notification target person and the robot, and the notification operation execution unit includes the execution information acquisition unit. When the familiarity in the familiarity information acquired in satisfies a predetermined familiarity condition, the informing operation is executed in cooperation with another robot.

(13) In the robot of the embodiment, the execution information acquisition unit acquires execution information associated with the plurality of notification targets, and the notification operation execution unit is associated with each of the notification targets. The notification operation is executed in parallel.

(14) Also, in the robot of the embodiment, the execution information acquisition unit acquires time information relating to the time for executing the notification operation as execution information, and the notification operation execution unit executes the notification operation based on the time information.

(15) The robot according to the embodiment further includes a voice recognition unit that recognizes voice input to the microphone and converts the voice into language data, and the execution information acquisition unit specifies execution information based on the converted language data. .

(16) In the robot according to the embodiment, the notifying operation execution unit may determine that when the obtained execution information does not include time information related to a time for executing the notifying operation, the predetermined time elapses from the time when the execution information is obtained. At that point, the notification operation is executed.

{Circle around (17)} In the robot of the embodiment, when the execution information does not include the location information, the execution information acquisition unit sets the location of the microphone to which the voice is input as the location information.

(18) The robot according to the embodiment further includes a photographing unit that photographs the notification target person after executing the notification operation, and a transmission unit that transmits the image data of the photographed notification target person to the user terminal.

(19) In the robot of the embodiment, the imaging unit changes the conditions for imaging the notification target person according to the intimacy between the notification target person and the robot.

(20) In the robot of the embodiment, a state information acquiring unit that acquires state information relating to the state of the person to be notified after executing the notifying operation, and a transmitting unit that transmits the acquired state information to the user terminal. Is further provided.

(21) In order to solve the above-described problem, the robot includes a message receiving unit that receives a message addressed to a user of the robot, a notification target specifying unit that specifies a notification target by a destination of the received message, and a message. A search unit that searches for a notification target person at a specified place or a place that is specified in correspondence with the notification target person, and the notification target person searched by the search unit is read out or instructed by a message. A notification unit for performing a notification operation.

(22) In order to solve the above-described problem, the robot control method according to the embodiment includes an execution information obtaining step of obtaining execution information for executing a notification operation of information to be performed on a notification target person in the robot; A search step of searching for a notification target person based on the execution information acquired in the execution information obtaining step; and a notification operation execution step of executing a notification operation based on the execution information for the notification target person searched in the search step. And

(23) In order to solve the above-described problem, the robot control program according to the embodiment includes an execution information acquisition process for acquiring execution information for performing an operation of informing a notification target person of information to be executed by a robot; A search process for searching for a notification target person based on the execution information acquired in the execution information acquisition process, and a notification operation execution process for executing a notification operation based on the execution information for the notification target person searched for in the search process And execute.

(24) In order to solve the above problem, the robot control method includes a message receiving step of receiving a message addressed to a user of the robot in the robot, and a notification target specifying the notification target by a destination of the received message. Step, a search step for searching for a notification target person at a place specified by the message or a place specified in correspondence with the notification target person, and, for the notification target person searched for in the search step, reading out a message or A notification step of performing a notification operation instructed by the message.

(25) In order to solve the above-described problem, the robot control program includes a message receiving process for receiving a message addressed to a user of the robot, and a notification target person specifying a notification target person by a destination of the received message. Processing, at a place specified by the message or at a place specified in correspondence with the notification target person, a search process for searching for the notification target person, and for the notification target person searched for in the search process, the reading of the message or And a notification process of performing a notification operation specified by the message.

According to one embodiment, a robot, a control method thereof, and a control program obtain execution information for performing a notification operation of information to be performed to a notification target person, and perform notification based on the obtained execution information. By searching for a target person and further performing a notification operation on the searched notification target person based on the execution information, it becomes possible to notify the information to a specific notification target person.

FIG. 2 is a block diagram illustrating an example of a software configuration of the autonomous behavior robot according to the embodiment. FIG. 2 is a block diagram illustrating an example of a hardware configuration of the autonomous behavior robot according to the embodiment. It is a flowchart which shows the 1st example of operation | movement of the autonomous behavior type robot control program in embodiment. 6 is a flowchart illustrating a second example of the operation of the autonomous behavior robot control program according to the embodiment. It is a flowchart at the time of the autonomous behavior type robot control program in an embodiment performing a wake-up operation as a notification operation. FIG. 4 is a diagram illustrating an example of execution information according to the embodiment. FIG. 6 is a diagram illustrating an example of a method for setting execution information according to the embodiment. It is a figure showing the example of module composition of the autonomous action type robot concerning execution information acquisition by voice input. It is a figure showing the example of module composition of the data providing device concerning specification of the attachment place of the attachment type device. FIG. 3 is a diagram illustrating an example of a module configuration of a data providing apparatus for specifying a location of a user terminal. FIG. 3 is a diagram illustrating an example of a module configuration of a data providing apparatus for specifying a location of a robot. FIG. 4 is a diagram illustrating an example of a module configuration of an autonomous behavior robot that provides a captured image and state information after executing a notification operation. FIG. 11 is a diagram illustrating a first specific example of a notification operation in which two robots cooperate. FIG. 14 is a diagram illustrating a second specific example of the notification operation in which two robots cooperate. FIG. 14 is a diagram illustrating a third specific example of the notification operation in which two robots cooperate. It is a flowchart which shows the alerting | reporting operation | movement by message reception.

Hereinafter, an autonomous robot of an embodiment of the present invention, a control method thereof, and a control program will be described in detail with reference to the drawings.

First, the software configuration of the autonomous behavior robot 1 will be described with reference to FIG. FIG. 1 is a block diagram illustrating an example of a software configuration of the autonomous behavior robot 1 according to the embodiment.

In FIG. 1, the autonomous behavior robot 1 has a data providing device 10 and a robot 2. The data providing device 10 and the robot 2 are connected by communication and function as the autonomous behavior robot 1. The robot 2 includes a photographing unit 21, a marker recognizing unit 22, a movement control unit 23, a state information acquisition unit 24, a search unit 25, a notification operation execution unit 26, a notification unit 27, and a movement mechanism 29 having respective functional units. It is. The data providing apparatus 10 has functional units of a first communication control unit 11, a point cloud data generation unit 12, a spatial data generation unit 13, a visualization data generation unit 14, an imaging target recognition unit 15, and a second communication control unit 16. . The first communication control unit 11 has functional units of a captured image acquisition unit 111, a spatial data providing unit 112, and an instruction unit 113. The second communication control unit 16 has functional units of a visualization data providing unit 161, a designation acquisition unit 162, and an execution information acquisition unit 163. The above-described functional units of the data providing device 10 of the autonomous behavior robot 1 according to the present embodiment will be described as being functional modules realized by a data providing program (software) that controls the data providing device 10. In addition, each function unit of the marker recognition unit 22, the movement control unit 23, the state information acquisition unit 24, the search unit 25, and the notification operation execution unit 26 of the robot 2 is controlled by a program that controls the robot 2 in the autonomous behavior type robot 1. The description will be made assuming that the function module is implemented.

The functions of the autonomous behavior robot 1 can be added by adding, deleting, or changing (adding, etc.) functional modules. The basic functions of the autonomous behavior robot 1 will be described as “basic functions”, and the additional functions of the autonomous behavior robot 1 will be described as “additional functions”. In the following description, the first communication control unit 11, the point cloud data generation unit 12, the spatial data generation unit 13, the visualization data generation unit 14, the imaging target recognition unit 15, and the second communication control unit 16 in the data providing device 10 The (visualization data providing unit 161, the designation acquisition unit 162), and the imaging unit 21, the marker recognition unit 22, and the movement control unit 23 of the robot 2 will be described as “basic functions”. Further, the execution information acquisition unit 163 in the data providing device 10 and the state information acquisition unit 24, the search unit 25, and the notification operation execution unit 26 in the robot 2 will be described as “additional functions”.

[Basic function]
The data providing device 10 is a device that can execute a part of the functions of the autonomous behavior robot 1. For example, the data providing device 10 is installed at a location physically close to the robot 2, communicates with the robot 2, It is an edge server that distributes the processing load. In the present embodiment, a case will be described in which the autonomous behavior robot 1 is configured by the data providing device 10 and the robot 2, but the function of the data providing device 10 is included in the function of the robot 2. Is also good. The robot 2 is a robot that can move based on spatial data, and is an embodiment of a robot whose moving range is determined based on spatial data. The data providing apparatus 10 may be configured in one housing or may be configured in a plurality of housings.

The first communication control unit 11 controls a communication function with the robot 2. The communication method with the robot 2 is arbitrary, and for example, a wireless LAN (Local Area Network), Bluetooth (registered trademark), short-range wireless communication such as infrared communication, or wired communication can be used. The functions of the captured image acquisition unit 111, the spatial data providing unit 112, and the instruction unit 113 included in the first communication control unit 11 communicate with the robot 2 using communication functions controlled by the first communication control unit 11.

The photographed image acquiring unit 111 acquires a photographed image photographed by the photographing unit 21 of the robot 2. The photographing unit 21 is provided in the robot 2 and can change the photographing range as the robot 2 moves. Here, the imaging unit 21, the marker recognition unit 22, the movement control unit 23, the state information acquisition unit 24, the search unit 25, the notification operation execution unit 26, the notification unit 27, and the movement mechanism 29 of the robot 2 will be described.

The photographing unit 21 photographs the space around the robot 2 and generates a photographed image including a space element. The space element is an element that exists in the space around the robot 2 and configures the space, and is, for example, a wall, a step, a door, furniture, home appliances, luggage, houseplants, and the like placed in the room. . The photographing unit 21 can be composed of one or a plurality of cameras. For example, when the photographing unit 21 is a stereo camera including two cameras, the photographing unit 21 can three-dimensionally photograph a spatial element to be photographed from different photographing angles. When the photographing unit 21 is constituted by a plurality of cameras, the photographed image may be a plurality of image data photographed by each camera or one image data obtained by combining the plurality of image data. The imaging unit 21 is a video camera using an imaging element such as a CCD (Charge-Coupled Device) sensor or a CMOS (Complementary Metal Oxide Semiconductor) sensor. By photographing a spatial element with two cameras (stereo cameras), the shape of the spatial element can be measured. Further, the photographing unit 21 may be a camera using a ToF (Time of Flight) technology. In a ToF camera, the shape of a spatial element can be measured by irradiating modulated infrared light to the spatial element and measuring the distance to the spatial element. Further, the photographing unit 21 may be a camera using a structured light. A structured light is a light that projects light of a stripe or lattice pattern onto a spatial element. The imaging unit 21 can measure the shape of the spatial element from the distortion of the projected pattern by imaging the spatial element from a different angle from the structured light. The photographing unit 21 may be any one of these cameras or a combination of two or more of them.

The photographing unit 21 is attached to the robot 2 and moves in accordance with the movement of the robot 2. However, the imaging unit 21 may be installed separately from the robot 2.

The captured image captured by the capturing unit 21 is provided to the captured image acquisition unit 111 in a communication method corresponding to the first communication control unit 11. The captured image is temporarily stored in the storage unit of the robot 2, and the captured image acquisition unit 111 acquires the temporarily stored captured image in real time or at a predetermined communication interval.

The marker recognizing unit 22 recognizes a predetermined marker included in the image captured by the image capturing unit 21. The marker is a space element indicating a restriction on movement of the robot 2. The movement restriction refers to restricting an operation associated with the movement of the robot 2, for example, restricting the movement speed of the robot 2, prohibiting the robot 2 from entering, or performing a predetermined operation of the robot 2 during the movement. (For example, generation of a sound from the robot 2). The marker is the shape, pattern or color of the article recognizable from the captured image or a combination thereof. When the user places a marker at a position where the movement of the robot 2 is restricted, when the robot 2 takes a picture of the space with the photographing unit 21, the picture is taken together with furniture and the like. The marker may be a planar article or a three-dimensional article. The marker is, for example, a sticker or paper on which a two-dimensional code or a specific color combination or shape is printed. The marker may be an ornament or a rug having a specific color or shape. In this way, by using printed matter or objects around us as markers, the user does not need to secure the power supply of the markers or secure the installation location. Further, the movement of the robot can be restricted by the user's intention without impairing the atmosphere of the room. In addition, since the user can visually recognize the marker, the user can intuitively grasp the movement restriction range, and can easily change the restriction range. The marker is set by the user, for example, by being attached to a wall or furniture, or placed on the floor. The marker recognizing unit 17 can recognize that the movement of the robot 2 is restricted by recognizing the marker image included in the captured image.

The marker recognizing unit 22 stores the visual characteristics of the marker in advance. For example, the marker recognizing unit 22 previously stores a two-dimensional code or a three-dimensional object to be recognized as a marker. The marker recognizing unit 22 may recognize an object registered in advance by a user as a marker. For example, when a user registers a flowerpot photographed by the camera of the user terminal 3 as a marker, the flowerpot installed in a corridor or the like can be recognized as a marker. Therefore, the user can install an object that does not cause discomfort at the place where the marker is installed as the marker. The marker recognizing unit 22 may recognize a spatial element other than an object as a marker. For example, the marker recognizing unit 22 may recognize a gesture of the user, such as a user crossing an arm in front of the body, as a marker. The marker recognizing unit 22 recognizes a position where the user makes a gesture as a marker installation position.

The marker recognizing unit 22 recognizes the position where the marker is attached or installed (hereinafter, referred to as “installation position”). The installation position is a position in the space where the marker in the space data is installed. The installation position can be recognized based on, for example, the spatial data recognized by the robot 2 based on the distance between the current position of the robot 2 and the captured marker. For example, when the size of the marker is known in advance, the marker recognizing unit 22 calculates the distance between the robot 2 and the marker from the size of the marker image included in the captured image, and determines the current position of the robot 2 and the capturing direction ( For example, the installation position of the marker can be recognized based on an azimuth (not shown). The installation position may be recognized from a relative position from a spatial element whose position in space is already known to a marker. For example, when the position of the door is already known, the marker recognizing unit 22 may recognize the installation position from the relative position of the marker and the door. Further, when the captured image is captured by a depth camera, the installation position can be recognized based on the depth of the marker captured by the depth camera.

The movement control unit 23 controls the movement mechanism 29. The movement control unit 23 can control the movement mechanism 29 based on the movement route (described later) calculated by the search unit 25. The movement control unit 23 controls the moving direction and the moving speed of the robot 2 by controlling the moving mechanism 29. The movement control unit 23 can recognize the current position of the robot 2 based on the spatial data recognized by the robot 2. The movement control unit 23 can move the robot 2 by controlling the movement mechanism 29 from the current position on the movement path. The movement control unit 23 may move the movement route by appropriately correcting the current position from a space element such as a wall or a corridor, for example.

The movement control unit 23 may control the movement mechanism 29 based on the marker recognized by the marker recognition unit 22. For example, the movement control unit 23 restricts movement by the movement mechanism 29 based on the installation position of the marker recognized by the marker recognition unit 22. The marker installation position includes a point, a line, a surface, or a space set based on one or more marker installation positions.

The spatial data providing unit 112 provides the robot 2 with the spatial data generated by the spatial data generating unit 13. The spatial data is data obtained by converting spatial elements recognized by the robot in the space where the robot 2 exists. The robot 2 can move within a range defined by the spatial data. That is, the spatial data functions as a map for determining a movable range in the robot 2. The robot 2 is provided with spatial data from the spatial data providing unit 112. For example, the spatial data can include position data of spatial elements such as walls, furniture, appliances, steps, etc., on which the robot 2 cannot move. The robot 2 can determine whether or not it is a place where it can move, based on the provided spatial data. Further, the robot 2 may be configured to be able to recognize whether or not an ungenerated range is included in the spatial data. Whether or not the ungenerated range is included can be determined, for example, based on whether or not a part of the spatial data includes a space having no spatial element.

The instructing unit 113 instructs the robot 2 to shoot based on the spatial data generated by the spatial data generating unit 13. The spatial data generating unit 13 creates spatial data based on the captured image acquired by the captured image acquiring unit 111. For example, when creating indoor spatial data, spatial data is not created for a part that is not imaged. May be included. Further, if the captured image is unclear or the like, the generated spatial data may include noise and the spatial data may include an inaccurate portion. If there is an ungenerated portion in the spatial data, the instruction unit 113 may issue a shooting instruction for the ungenerated portion. In addition, when the space data includes an inaccurate portion, the instructing unit 113 may instruct the imaging of the inaccurate portion. The instruction unit 113 may spontaneously instruct shooting based on the spatial data. Note that the instruction unit 113 may instruct shooting based on an explicit instruction from a user who has confirmed visualization data (described later) generated based on spatial data. The user can specify a region included in the visualization data and instruct the robot 2 to shoot, thereby recognizing a space and generating space data.

The instructing unit 113 may instruct to shoot a marker set in the area. The shooting in the area where the creation of the spatial data is instructed may include, for example, shooting conditions such as the coordinate position of the robot 2 (the shooting unit 21), the shooting direction of the shooting unit 21, and the resolution. When the spatial data instructed to be created relates to an ungenerated area, the spatial data generating unit 13 adds the newly created spatial data to the existing spatial data. If the designated spatial data is for re-creation, the spatial data is generated by updating the existing spatial data. When a marker is included in the captured image, spatial data including the recognized marker may be generated.

The point cloud data generation unit 12 generates three-dimensional point cloud data of a spatial element based on the captured image acquired by the captured image acquisition unit 111. The point cloud data generation unit 12 generates point cloud data by converting a spatial element included in the captured image into a set of three-dimensional points in a predetermined space. The space elements are, as described above, the walls, steps, doors, furniture, home appliances, luggage, houseplants, and the like placed in the room. Since the point cloud data generation unit 12 generates the point cloud data based on the captured image of the space element, the point cloud data represents the shape of the surface of the captured space element. The photographed image is generated by the photographing unit 21 of the robot 2 photographing at a predetermined photographing position at a predetermined photographing angle. Therefore, when the robot 2 photographs a spatial element such as furniture from the front position, point cloud data cannot be generated for the shape of the back side of the furniture that is not photographed, and the robot 2 moves to the back side of the furniture. Even if there is a possible space, the robot 2 cannot recognize it. On the other hand, when the robot 2 moves and photographs furniture from the photographing position on the side surface, point cloud data can be generated for the shape on the back side of a space element such as furniture, so that the space can be correctly grasped.

The space data generation unit 13 generates space data that defines a movable range of the robot 2 based on the point cloud data of the space elements generated by the point cloud data generation unit 12. Since spatial data is generated based on point cloud data in space, spatial elements included in spatial data also have three-dimensional coordinate information. The coordinate information may include information on the position, length (including height), area, or volume of the point. The robot 2 can determine the movable range based on the position information of the spatial element included in the generated spatial data. For example, when the robot 2 has the moving mechanism 29 that horizontally moves on the floor, the robot 2 is configured such that the step from the floor, which is a spatial element in the spatial data, is equal to or more than a predetermined height (for example, 1 cm or more). In some cases, it can be determined that movement is impossible. On the other hand, in the spatial data, when the table top or bed or the like, which is a spatial element, has a predetermined height from the floor, the robot 2 determines that the height from the floor is higher than the predetermined height (for example, 60 cm or more). The range of ()) is determined as a movable range in consideration of the clearance with its own height. In addition, the robot 2 determines, in the spatial data, a range in which the gap between the wall and the furniture, which is a space element, is equal to or greater than a predetermined width (for example, 40 cm or more) as a movable range in consideration of a clearance between itself and the width. I do.

The space data generation unit 13 may set attribute information for a predetermined area in the space. The attribute information is information that defines a moving condition of the robot 2 for a predetermined area. The moving condition is, for example, a condition that defines a clearance with a space element to which the robot 2 can move. For example, when a normal moving condition under which the robot 2 can move has a clearance of 30 cm or more, attribute information in which the clearance for a predetermined area is exceptionally 5 cm or more can be set. Further, as the movement condition set in the attribute information, information for restricting movement of the robot may be set. The movement restriction is, for example, a restriction on a moving speed or a prohibition of entry. For example, attribute information in which the moving speed of the robot 2 is reduced may be set in an area having a small clearance or an area where a person exists. The moving condition set in the attribute information may be determined by the floor material in the area. For example, the attribute information may be for setting a change in the operation (running speed or running means, etc.) of the moving mechanism 29 when the floor is a cushion floor, flooring, tatami, or carpet. In addition, the attribute information may be set such that a charging spot at which the robot 2 can be moved and charged, a step at which movement of the robot 2 is restricted because the posture of the robot 2 becomes unstable, or a moving condition at a carpet edge or the like can be set. . The area in which the attribute information is set may be configured so that the user can grasp the area, for example, by changing the display method in the visualization data described later.

The spatial data generation unit 13 performs, for example, Hough transform on the point cloud data generated by the point cloud data generation unit 12 to extract a graphic such as a straight line or a curve common to the point cloud data. Spatial data is generated according to the contour of the represented spatial element. The Hough transform is a coordinate transformation method for extracting a figure which passes through the feature points most, when the point group data is a feature point. Since the point cloud data expresses the shape of a space element such as furniture placed in a room in a point cloud, the user determines which space element is represented by the point cloud data (for example, Recognition of tables, chairs, walls, etc.) can be difficult. The spatial data generation unit 13 can represent the outline of furniture or the like by Hough transforming the point cloud data, so that the user can easily determine the spatial element. The space data generation unit 13 converts the point cloud data generated by the point cloud data generation unit 12 into a basic shape of a space element (for example, a table, a chair, a wall, or the like) recognized in the image recognition, and converts the space into a basic shape. Data may be generated. When a spatial element such as a table is recognized as a table by image recognition, the shape of the table is converted from a part of the point group data of the spatial element (for example, point cloud data when the table is viewed from the front). It can be accurately predicted. By combining the point cloud data and the image recognition, the spatial data generating unit 13 can generate spatial data in which spatial elements are accurately grasped.

The space data generation unit 13 generates space data based on point cloud data included in a predetermined range from the position where the robot 2 has moved. The predetermined range from the position where the robot 2 has moved includes the position where the robot 2 has actually moved, and may be, for example, a range having a distance of 30 cm or the like from the position where the robot 2 has actually moved. Since the point cloud data is generated based on an image captured by the image capturing unit 21 of the robot 2, the captured image may include a spatial element at a position distant from the robot 2. When the space element is separated from the imaging unit 21, there may be a part where the robot 2 cannot actually move due to the presence of a part that has not been captured or the presence of an obstacle that has not been captured. Further, when a spatial element at a position far from the photographing unit 21 such as a corridor is included in the photographed image, the spatial element extracted at the feature point may be distorted. Further, when the shooting distance is long, the spatial element included in the shot image is small, and the accuracy of the point cloud data may be low. The spatial data generating unit 13 may generate spatial data that does not include a low-accuracy spatial element or a distorted spatial element by ignoring feature points that are far apart. The spatial data generation unit 13 deletes point cloud data outside a predetermined range from the position to which the robot 2 has moved to generate spatial data, thereby preventing an enclave where no data actually exists from occurring. Therefore, it is possible to generate spatial data that does not include a range in which the robot 2 cannot move and has high data accuracy. Also, in the visualization data generated from the spatial data, engraved drawing can be prevented, and the visibility can be improved.

When the marker recognition unit 22 recognizes a marker, the spatial data generation unit 13 sets a limited range for the generated spatial data. By setting a limit range for the spatial data, the limit range can be visualized as a part of the visualization data. When the state information is acquired by the state information acquiring unit 24, the spatial data generating unit 13 sets the state information for the spatial data. By setting the state information for the spatial data, the state information can be made a part of the visualization data.

The visualization data generation unit 14 generates visualization data based on the spatial data generated by the spatial data generation unit 13 so that a person can intuitively determine a space element included in the space.

ロボット Generally, a robot has various sensors such as a camera and a microphone, and recognizes surrounding conditions by comprehensively judging information obtained from those sensors. In order for the robot to move, it is necessary to recognize various objects existing in the space and determine the movement route in the spatial data. However, the movement route may not be appropriate because the object cannot be recognized correctly. Due to erroneous recognition, for example, even if a person thinks that there is a sufficiently large space, the robot may recognize that there is an obstacle and that the robot can move only in a small area. When a discrepancy in recognition occurs between the human and the robot in this way, the robot performs an action that is contrary to human expectations, and the human feels stress. The autonomous behavior robot 1 in the present embodiment visualizes and provides spatial data, which is its own recognition state, to a person in order to reduce inconsistencies between the recognition of the person and the robot, and also provides a method for recognizing a part pointed out by the person. The recognition process can be performed again.

The spatial data is data including a spatial element recognized by the autonomous robot 1, whereas the visualization data is data for allowing the user to visually recognize the spatial element recognized by the autonomous robot 1. Data. The spatial data may include a misrecognized spatial element. By visualizing the spatial data, it becomes easier for a person to confirm the recognition state (whether or not there is an erroneous recognition) of the spatial element in the autonomous robot 1.

Visualization data is data that can be displayed on a display device. The visualization data is a so-called floor plan, and includes a space element recognized as a table, a chair, a sofa, or the like in an area surrounded by the space element recognized as a wall. The visualization data generation unit 14 generates a shape of furniture or the like formed in the figure extracted by the Hough transform as, for example, visualization data represented by RGB data. The spatial data generation unit 13 generates visualization data in which the drawing method of the plane is changed based on the direction of the three-dimensional plane of the spatial element. The direction of the three-dimensional plane of the spatial element is, for example, the direction of the normal to the plane formed by the figure generated in the point cloud data by subjecting the point cloud data generated by the point cloud data generation unit 12 to Hough transform. It is. The visualization data generation unit 14 generates visualization data in which the plane drawing method is changed according to the normal direction. The drawing method is, for example, a color attribute such as hue, lightness or saturation to be given to a plane, a pattern to be given to a plane, or a texture. For example, when the normal of the plane is in the vertical direction (the plane is the horizontal direction), the visualization data generation unit 14 increases the brightness of the plane and draws the plane in a bright color. On the other hand, when the normal of the plane is in the horizontal direction (the plane is in the vertical direction), the visualization data generation unit 14 renders the plane with a lower brightness and draws in a dark color. By changing the drawing method of the plane, the shape of furniture and the like can be expressed three-dimensionally, and the user can easily check the shape of furniture and the like. Further, the visualization data may include coordinate information in the visualization data associated with the coordinate information of each space element included in the space data (referred to as “visualization coordinate information”). Since the visualized coordinate information is associated with the coordinate information, a point in the visualized coordinate information corresponds to a point in an actual space, and a surface in the visualized coordinate information corresponds to a surface in an actual space. Therefore, when the user specifies the position of a certain point in the visualization data, the corresponding position of the point in the actual room can be specified. Further, a conversion function for converting the coordinate system may be prepared so that the coordinate system in the visualization data and the coordinate system in the spatial data can be mutually converted. Of course, the coordinate system in the visualization data and the coordinate system in the actual space may be interchangeable.

The visualization data generation unit 14 generates visualization data as three-dimensional (3D (Dimensions)) data. In addition, the visualization data generation unit 14 may generate the visualization data as two-dimensional (2D) data. By generating the visualization data in 3D, the user can easily check the shape of furniture and the like. The visualization data generation unit 14 may generate the visualization data in 3D when the spatial data generation unit 13 generates enough data to generate the visualization data in 3D. The visualization data generation unit 14 may generate the visualization data in 3D based on the 3D viewpoint position (viewpoint height, viewpoint elevation / depression angle, etc.) specified by the user. By making it possible to specify the viewpoint position, it is possible for the user to easily check the shape of furniture or the like. In addition, the visualization data generation unit 14 may generate visualization data in which only the back wall or ceiling of the room is colored, and the front wall or ceiling is transparent (not colored). By making the wall on the near side transparent, it is possible for the user to easily check the shape of furniture and the like arranged at the end (in the room) of the wall on the near side.

The visualization data generation unit 14 generates visualization data to which a color attribute according to the captured image acquired by the captured image acquisition unit 111 is added. For example, when the captured image includes woodgrain furniture and detects a woodgrain color (for example, brown), the visualization data generation unit 14 assigns a color similar to the detected color to the extracted furniture graphic. Generate visualization data. By giving the color attribute according to the captured image, it is possible for the user to easily confirm the type of the furniture or the like.

(4) The visualization data generation unit 14 generates visualization data in which a drawing method of a fixed object that is fixed and a moving object that moves is changed. The fixed object is, for example, a room wall, a step, fixed furniture, or the like. The moving object is, for example, a chair, a trash can, furniture with casters, or the like. In addition, the moving object may include, for example, temporary objects such as luggage and bags temporarily placed on the floor. The drawing method is, for example, a color attribute such as hue, lightness or saturation to be given to a plane, a pattern to be given to a plane, or a texture.

区分 The classification of fixed objects, moving objects or temporary objects can be identified by the period of their existence at the place. For example, the spatial data generation unit 13 identifies the space element as a fixed object, a moving object, or a temporary object based on a change with time of the point cloud data generated by the point cloud data generation unit 12 and converts the space data. Generate. For example, the spatial data generation unit 13 determines that the spatial element is a fixed object when the spatial element has not changed from the difference between the spatial data generated at the first time and the spatial data generated at the second time. to decide. Further, the spatial data generation unit 13 may determine from the difference in the spatial data that the spatial element is a moving object when the position of the spatial element has changed. Further, the spatial data generation unit 13 may determine from the difference in the spatial data that the spatial element is a temporary when the spatial element has disappeared or has appeared. The visualization data generation unit 14 changes the drawing method based on the section identified by the spatial data generation unit 13. The change of the drawing method is, for example, color coding, addition of hatching, addition of a predetermined mark, or the like. For example, the spatial data generation unit 13 may display a fixed object in black, display a moving object in blue, or display a temporary object in yellow. The spatial data generating unit 13 generates spatial data by identifying a fixed object, a moving object, or a temporary object. The visualization data generation unit 14 may generate the visualization data in which the drawing method is changed based on the section identified by the spatial data generation unit 13. Further, the spatial data generation unit 13 may generate visualization data in which the rendering method of the spatial element recognized by the image recognition is changed.

The visualization data generation unit 14 can generate visualization data in a plurality of divided areas. For example, the visualization data generation unit 14 generates the visualization data with a space partitioned by walls such as a living room, a bedroom, a dining room, and a corridor as one room. By generating the visualization data for each room, for example, the generation of the spatial data or the visualization data can be performed separately for each room, and the generation of the spatial data or the like becomes easy. Further, it is possible to create spatial data and the like only for an area where the robot 2 may move. The visualization data providing unit 161 provides visualization data from which a user can select an area. The visualization data providing unit 161 may, for example, enlarge the visualization data of the area selected by the user or provide detailed visualization data of the area selected by the user.

The photographing target recognizing unit 15 recognizes a spatial element based on the photographed image acquired by the photographed image acquiring unit 111. Recognition of a spatial element can be performed, for example, by using an image recognition engine that determines what the spatial element is based on image recognition results accumulated in machine learning. The image recognition of the spatial element can be performed based on, for example, the shape, color, and pattern of the spatial element. The imaging target recognition unit 15 may be configured to be able to perform image recognition of a spatial element by using, for example, an image recognition service provided in a cloud server (not shown). The visualization data generation unit 14 generates visualization data in which the drawing method is changed according to the spatial element whose image has been recognized by the imaging target recognition unit 15. For example, when the spatial element whose image has been recognized is a sofa, the visualization data generation unit 14 generates visualization data in which a texture having a texture of cloth is added to the spatial element. When the space element whose image has been recognized is a wall, the visualization data generation unit 14 may generate visualization data to which a color attribute of wallpaper (for example, white) is added. By performing such visualization processing, the user can intuitively grasp the recognition state of the space in the robot 2.

The second communication control unit 16 controls communication with the user terminal 3 owned by the user. The user terminal 3 is, for example, a smartphone, a tablet PC, a notebook PC, a desktop PC, or the like. The communication method with the user terminal 3 is arbitrary, and for example, wireless LAN, Bluetooth (registered trademark), short-range wireless communication such as infrared communication, or wired communication can be used. Each function of the visualization data providing unit 161, the designation acquisition unit 162, and the execution information acquisition unit 163 included in the second communication control unit 16 communicates with the user terminal 3 using the communication function controlled by the second communication control unit 16. connect.

The visualization data providing unit 161 provides the visualization data generated by the visualization data generation unit 14 to the user terminal 3. The visualization data providing unit 161 is, for example, a Web server, and provides the browser of the user terminal 3 with the visualization data as a Web page. The visualization data providing unit 161 may provide the visualization data to a plurality of user terminals 3. By visually recognizing the visualization data displayed on the user terminal 3, the user can confirm the range in which the robot 2 can move as a 2D or 3D display. In the visualization data, shapes of furniture and the like are drawn by a predetermined drawing method. By operating the user terminal 3, the user can, for example, switch between 2D display and 3D display, zoom in or out on the visualized data, or move the viewpoint in 3D display.

(4) The user can visually recognize the visualized data displayed on the user terminal 3 and check the generation state of the spatial data and the attribute information of the area. The user can designate a region in which no spatial data has been generated from the visualized data and instruct creation of the spatial data. Further, the user visually recognizes the visualized data displayed on the user terminal 3 and, if there is an area in which the spatial data is considered to be inaccurate, such as an unnatural shape of a space element such as furniture, is displayed. A region can be designated to instruct the regeneration of spatial data. As described above, since the visualized coordinate information in the visualized data is associated with the coordinate information of the spatial data, the region in the visualized data designated to be regenerated by the user can be uniquely specified as the region in the spatial data. . The visualization data is regenerated in the visualization data generation unit 14 based on the regenerated spatial data, and is provided from the visualization data providing unit 161. It should be noted that the generated state of the spatial data may not change even if the spatial element is erroneously recognized even in the regenerated visualized data. In this case, the user may instruct the generation of the spatial data by changing the operation parameter of the robot 2. The operation parameters include, for example, photographing conditions (exposure amount or shutter speed) of the photographing unit 21 of the robot 2, sensitivity of a sensor (not shown), clearance conditions for permitting the movement of the robot 2, and the like. The operation parameters may be included in the spatial data as area attribute information, for example.

The visualization data generation unit 14 generates, for example, visualization data including display of a button for instructing creation (including “re-creation”) of spatial data. The user terminal 3 can transmit an instruction for creating spatial data to the autonomous robot 1 by operating the displayed button by the user. The instruction to create spatial data transmitted from the user terminal 3 is acquired by the designation acquisition unit 162.

The designation acquisition unit 162 acquires an instruction to create spatial data of the area designated by the user based on the visualization data provided by the visualization data providing unit 161. The designation acquisition unit 162 may acquire an instruction to set (including change) the area attribute information. In addition, the designation acquisition unit 162 acquires the position of the region and the direction when the robot approaches the region, that is, the direction to be photographed. Acquisition of the creation instruction can be executed, for example, by operating the Web page provided by the visualization data providing unit 161. Thereby, the user can grasp how the robot 2 recognizes the space, and can instruct the robot 2 to perform the recognition process again according to the recognition state.

[Additional functions]
The state information acquisition unit 24 acquires state information relating to the state of the notification target person searched by the search unit 25. The state of the notification target person is the state of the motion of the notification target person, and is, for example, a sleeping state, a rising state, a sitting state, a standing state, a walking state, or cooking or cleaning. And the like during operation. As the state of the notification target person, a health state or a mental state of the notification target person may be acquired. The state information acquisition unit 24 determines the state of the notification target person based on, for example, a captured image of the notification target person captured by the imaging unit 21. The state information acquisition unit 24 is a sound emitted by the notification target person collected by a microphone (not shown), a heat distribution of the notification target person measured by the radiation thermometer, the illuminance of the room measured by the illuminometer, and detected by the proximity sensor. The state of the notification target person may be determined based on the movement of the notification target person, the heart rate acquired from the heart rate meter worn by the notification target, and the like. For example, the state information acquisition unit 24 may determine the sleep state of the notification target person from these pieces of information. The sleep state is, for example, a state such as REM sleep, non-REM sleep, and the depth of sleep that can be measured from a person's heart rate, body movement, and the like.

For example, the state information acquisition unit 24 determines in advance the captured image of the notification target person captured by the imaging unit 21, the heat distribution of the notification target person, or the movement of the notification target person in the notification target person detected by the proximity sensor. When it is determined that the movement is small during the set time and the operation range, the notification target person is determined to be in a sleeping state. In addition, the state information acquisition unit 24 determines whether the movement of the notification target person is the movement amount of the notification target person (for example, the integrated value or the average value of the movement amount of the notification target person per time) in a predetermined time and operation range. Is larger than the predetermined amount of movement, the notification target person may be determined to be in the state of getting up. The predetermined operation amount may be, for example, a value determined by an experiment or the like. The state information acquisition unit 24 may determine that the notification target person is in a sitting state from the posture of the notification target person or the positional relationship between a spatial element such as a chair and the notification target person. Further, the state information acquisition unit 24 may determine that the notification target person is in a standing state from the posture of the notification target person. Further, when the position of the notification target person in the space is moving, the state information acquisition unit 24 may determine that the notification target person is in a state of walking. Further, the state information acquisition unit 24 may determine the work content of the notification target person from the comparison between the operation of the notification target person in the captured image and a previously learned operation pattern, and determine that the notification target person is working.

The state information acquisition unit 24 may determine the state of the notification target person based on the lighting state of the lighting and the open / closed state of the door. For example, when the state information acquisition unit 24 determines that the illumination is turned off from the illuminance of the room measured by the illuminometer, the notification target person may determine that the notification target person is in a sleeping state.

The search unit 25 searches for a notification target person based on the execution information acquired by the execution information acquisition unit 163. The notification operation execution unit 26 performs a notification operation on the notification target person searched by the search unit 25 based on the execution information.

Execution information is information for executing a notification operation of information to be performed to a notification target person, and is information set for the autonomous behavior robot 1 by a user. The execution information includes, for example, information of a notification target person for specifying a notification target person to perform the notification operation, information of a place to search the notification target person, information of the notification operation, and information of a time at which the notification operation is performed. Including. The execution information is set in advance by a user, for example, and provided to the execution information acquisition unit 163.

<Information of target person>
The information of the notification target is information for identifying the notification target, and includes, for example, the name of the notification target, ID (Identification), information indicating physical characteristics, personal belongings, clothing, or the robot 2. Such as intimacy (described later). The information indicating the physical characteristics of the notification target includes, for example, information for recognizing the face of the notification target (face recognition), information for recognizing the fingerprint of the notification target (fingerprint recognition), and the notification target. This is information for recognizing the physique of the person (shape recognition). The information on the personal belongings, clothing, and the like of the notification target person is, for example, information on a wireless tag owned by the notification target person, information indicating the characteristics of the notification target person's clothing, and the like. The search unit 25 can specify the notification target person by determining whether or not the notification target person is based on the information of the notification target person. For example, when a plurality of people are present in one room, the search unit 25 can specify the notification target person based on the information of the notification target person. The information of the notification target person may be stored in advance in the autonomous behavior robot 1 together with an ID for specifying the notification target person, and only the ID may be specified in the execution information.

(4) The execution information may include information of one or more notification target persons. By including information on a plurality of notification targets in the execution information, the robot 2 can sequentially execute a notification operation on a plurality of notification targets. When the information of the notification target includes the notification target, a notification operation set in advance for each notification target is executed. Further, the execution order (priority) of the notification operation for a plurality of notification target persons may be set in the execution information.

<Search location information>
The information on the location (search location) for searching for the notification target is information (location information) of a place where the notification target is expected to exist. For example, the robot 2 moves to search for the notification target. It is a destination to move. The location information is, for example, position information indicating a point, line, or range in the space, or information on a room in which the position information in the space is registered in advance. The search unit 25 calculates a moving route by the moving mechanism 29 based on the location information included in the execution information. The movement route can be calculated from the current position of the robot 2 and the search location. For example, the search unit 25 can store a range in which the moving mechanism 29 can move in advance, and calculate a moving route that can move from the current position to the search location in the shortest distance in the movable range. The search unit 25 may calculate the moving speed in the moving route by including the moving speed in the moving route. For example, the search unit 25 can calculate the moving route so as to move while changing the moving speed in the hallway and the moving speed in the room. The movement control unit 23 controls the movement mechanism 29 based on the movement path calculated by the search unit 25 to move the robot 2.

The search unit 25 may calculate the movement route based on the movement restriction range in which the movement of the marker recognized by the marker recognition unit 22 is restricted. For example, when the entry prohibition range is set based on the marker, the search unit 25 calculates the moving route so as to avoid the entry prohibition range. When the movement speed is limited based on the marker, the search unit 25 may calculate the movement route so that the movement taking the speed limit into consideration is the shortest time.

The location information may be specified by the user. For example, the user specifies location information by specifying a search location on a map displayed on the user terminal 3 operated by the user. The execution information acquisition unit 163 can acquire the location information specified by the user from the user terminal 3. The search unit 25 calculates a moving route by the moving mechanism 29 based on the location information specified by the user from the user terminal 3. The search unit 25 calculates, for example, a movement path from the home position where the robot 2 returns for charging to the search location. The search unit 25 may display an alert on the user terminal 3 when there is a marker, a stair, or the like that restricts movement on the calculated movement route. That is, the search unit 25 may calculate the movement path when the robot 2 moves to the search place (immediately before or during movement), or calculates the movement path when the execution information is set by the user. You may do so.

The execution information may include information on one or more search locations. By including the information of a plurality of search locations in the execution information, the robot 2 can, for example, if the notification target cannot be searched (discovered) at the first specified search location, the robot 2 can be notified at the other specified search location. Can be searched.

<Information of notification operation>
The information on the notification operation is information indicating the content of the notification operation performed on the notification target person. The notifying operation executing unit 26 executes the notifying operation on the notification target person searched by the searching unit 25 based on the information of the notifying operation. The notification operation is an operation of notifying the notification target person of the notification information using the notification unit 27. That is, the notification operation execution unit 26 can execute the notification operation via the notification unit 27. The information of the notification operation includes time information at which the notification operation is performed. The search unit 25 searches for a notification target person according to the time information.

The notification unit 27 is an output device such as a speaker, a display, or an actuator. The loudspeaker reports information on the hearing of the person to be reported by sound (including voice). The display is, for example, a display, a light, or the like, and notifies information to the notification target person's vision by information (character, image, light, or the like) displayed on the display. The actuator is, for example, a movable unit such as a robot hand, a vibration generator, or a compressed air output valve, and notifies information on the tactile sensation of the notification target person. The notifying unit 27 may notify information on the sense of smell or taste of the notification target person. The notification operation is, for example, output of a sound from a speaker to a notification target person, output of display information from a display, or contact of the notification target person with a robot hand. The notification operation may be a combination of these notification operations. The notification operation can be specified by the user. For example, the notification operation may specify a purpose of notification such as “alarm operation” or “time notification operation”. The notification operation execution unit 26 may execute the notification operation according to the information of the notification target person. For example, when the goodness of getting up of the person to be notified is stored as the information of the person to be notified, the notifying operation executing unit 26 executes the notifying operation according to the goodness of getting up of the person to be notified. The goodness of getting up can be evaluated based on the time from when the notification operation is performed to when the state of the notification target person becomes other than the sleep state. For example, as the information of the notification target person, the notification target person wakes up poorly (for example, the average value of the time from when the notification operation is performed to when the state of the notification target person becomes other than the sleep state is larger than the wake-up threshold). When the notification is stored, the notification operation execution unit 26 may execute the notification operation with a sound of a predetermined volume or more. In addition, instead of or in addition to increasing the volume, the notification operation execution unit 26 may control so as to increase the time for outputting the sound. Alternatively, the notification operation executing unit 26 may control the type of sound to be the first type of sound instead or in addition to the above. Alternatively, the notification operation executing unit 26 may control the increase of the volume per time to be relatively large instead of or in addition to the above. Also, when it is stored that the notification target person is good to wake up (for example, the average value of the time from when the notification operation is performed until the state of the notification target person becomes other than the sleep state is smaller than the wake-up threshold). The notifying operation executing unit 26 may execute the notifying operation with a sound of a predetermined volume or less so as not to surprise the notification target person. In addition, instead of or in addition to increasing the volume, the notification operation execution unit 26 may perform control so as to shorten the time for outputting the sound. Alternatively, the notification operation execution unit 26 may control the type of sound to be a second type of sound different from the first type of sound instead or in addition to the above. In addition, the notification operation executing unit 26 may control the increase of the volume per time to be relatively small instead of or in addition to the above. In addition, the notification operation execution unit 26 may change the notification operation according to the intimacy with the notification target person (described later), or may change the notification operation based on past data.

[Alarm operation]
The “alarm operation” is a notification operation for waking up a notification target person who is sleeping during sleep based on time information. In the alarming operation of “alarm”, the alarming operation executing unit 26 outputs an alarming sound from the speaker of the alarming unit 27, displays the current time on a display, or contacts the alarming person by a robot arm to notify the alarm. Wake up the subject. The notification operation executing unit 26 causes the alarm target operation to be executed when the notification target person searched by the search unit 25 is sleeping. Whether or not the notification target person is sleeping can be determined by the state information acquisition unit 24. The notification operation execution unit 26 may repeat the wake-up operation until the notification target person wakes up. Whether or not the notification target person has woken up can be determined by the state information acquisition unit 24. It should be noted that the notification target person may wake up not to sleep twice (being once awake and then sleeping again). The notification operation execution unit 26 may repeat the wake-up operation when the state information acquisition unit 24 determines that the user has slept twice.

[Time notification operation]
The “time notification operation” is an operation of notifying a notification target person of a time designated in advance. The time notification operation for the notification target person is, for example, an operation of notifying the start time of the television broadcast. The notification operation execution unit 26 may output a voice prompting the notification target person to view the television at a preset time. In addition, the notification operation execution unit 26 may turn on the power of the television at a preset time. The notification operation execution unit 26 may record a television program when the notification target person is not watching television. Whether or not the notification target is watching the television can be determined by the state information acquisition unit 24 by determining whether or not the notification target is sitting at the place where the television is viewed.

時刻 The time notification operation for the notification target person may be an operation for notifying the notification target person of the leaving time. The notification operation execution unit 26 may output a voice prompting the notification target person to go out when a preset time comes. In addition, the notification operation execution unit 26 may execute the door closing when the notification target person goes out after a preset time. Whether or not the notification target person has gone out can be determined by the state information acquisition unit 24 determining that the person has left the entrance.

<Completion condition of notification operation>
The notification operation information may include a notification operation completion condition. The notification operation completion condition is a condition for regarding that the notification operation for the notification target has been completed. Further, the completion condition of the notification operation may include an operation when the completion condition is not satisfied. For example, when the completion condition is not satisfied, the notification operation execution unit 26 may continue or repeat the notification operation until the completion condition is satisfied. By including the completion condition in the information of the notification operation, it becomes possible to execute the notification operation according to the situation of the notification target person. The completion condition of the notification operation may be provided to the autonomous behavior robot 1 as information of the notification operation, or may be set in the autonomous behavior robot 1 in advance. The following is an example of a completion condition when the notification operation is a wake-up operation.

For example, the completion condition may be “when it is determined that the notification target person has woken up at the search location”. When it is determined that the notification target person has woken up at the search location, the notification operation has been achieved, and thus the notification operation can be completed. Whether or not the notification target person has woken up can be determined by the state information acquisition unit 24 as described above.

完了 Further, the completion condition may be “when the notification target person cannot be searched at the search place”. When the notification target person cannot be searched at the search location, the notification target person can be regarded as having already woken up, and thus the notification operation can be completed. Whether or not the notification target person has been searched at the search place can be determined by the state information acquisition unit 24.

(4) The completion condition may be “when a notification target person is found in a place other than the search place”. The place other than the search place is, for example, a place on the movement route of the robot 2. When the notification target person is found in a place other than the search place, the notification target person can be regarded as having already woken up, so that the notification operation can be completed. The discovery of the notification target person can be performed, for example, by the state information acquisition unit 24 performing image recognition of the notification target person with respect to the captured image, or voice recognition of the voice to the robot 2 from the notification target person. .

完了 Also, the completion condition may be “when a pleasant action from the notification target person is detected”. The pleasant action is an action of the notification target person of a predetermined type. For example, the notification target person “strokes” the robot 2 or “acts to thank” or “greets” the robot 2. Act. When a pleasant action from the notification target person is detected, the notification target person can be regarded as having already woken up, so that the notification operation can be completed. The detection of a pleasant act from the notification target person can be performed by, for example, the state information acquisition unit 24.

<Responsibility level of notification operation>
The information of the notification operation may include a responsibility level of the notification operation. The responsibility level of the notification operation is information indicating the importance of the notification operation to the notification target person. For example, when the responsibility level is high, the notification operation execution unit 26 repeatedly executes the notification operation until the state of the notification target person becomes a predetermined state. For example, when the responsibility level is high in the wake-up operation, the notification operation execution unit 26 continues the wake-up operation until the notification target person wakes up. When the responsibility level is high, the robot 2 may start searching for the notification target person early so that the wake-up operation can be reliably performed at the set time. When the responsibility level is a medium level lower than the high level, the notification operation executing unit 26 may execute the wake-up operation a preset number of times. When the responsibility level is the middle level, the robot 2 may start searching for the notification target person before a predetermined time from the set time. When the responsibility level is lower than the middle level, the notification operation executing section 26 may execute the wake-up operation only once. When the responsibility level is low, the robot 2 may start searching for a notification target person after execution of another notification operation (for example, a wake-up operation for another notification target) ends.

<Intimacy between the notification target and the robot>
The intimacy between the notification target person and the robot 2 is information obtained by indexing the subjective feeling that the notification target person feels about the robot 2. For example, the notification target person may have a feeling of familiarity with the robot 2 depending on the shape of the robot 2 (for example, a shape of a human or an animal), a voice output by the robot 2, an operation of the robot 2, and the like. . In addition, the notification target person may have a familiar feeling based on the experience of past contact with the robot 2 or the like. The intimacy level is expressed, for example, as a percentage (0 to 100%) or a plurality of levels (S, A, B, and C levels) based on the emotion of the notification target person. The autonomous behavior robot 1 may store the intimacy and update it in accordance with the past actions and opinions of the notification target person. For example, the autonomous behavior robot 1 may be updated so as to increase intimacy when the notification target performs a pleasant act on the robot 2.

場合 When there are a plurality of notification targets, the intimacy may be set for each notification target. When a plurality of robots 2 exist, the intimacy level may be set for each robot 2. For example, when there is one notification target person A and two robots 2 including the robots 2a and 2b, each of the robots 2a and 2b can set the degree of intimacy with the notification target person A.

The notification operation based on the execution information may be performed according to the intimacy between the notification target person and the robot 2. For example, in the above-described notification operation to the notification target person A, while the robot 2a having a high degree of closeness performs the notification operation including contact with the notification target person, the robot 2b having a low degree of closeness outputs a voice output to the notification target person. May be executed only. In addition, when the notification operation is set for the robot 2b with low intimacy, the robot 2a with high intimacy may execute the notification operation in cooperation with the robot 2b or instead of the robot 2. When both the intimacy of the robot 2a and the intimacy of the robot 2b are high, the notification operation may be performed such that the robot 2a and the robot 2b cooperate or compete with each other. By performing the notification operation according to the intimacy level, it is possible to execute the notification operation according to the notification target person.

The execution information acquisition unit 163 acquires execution information for performing a notification operation of information to be performed to a notification target person. The execution information acquisition unit 163 can acquire execution information from the user terminal 3. The execution information acquisition unit 163 may acquire execution information by receiving execution information transmitted from the user terminal 3 by an operation of the user who operates the user terminal 3. Further, the execution information acquisition unit 163 may acquire the execution information by downloading the execution information stored in the storage unit of the user terminal 3. Further, the execution information acquisition unit 163 may acquire the execution information from a data server (not shown).

Further, the execution information acquisition unit 163 is configured to operate the map displayed on the user terminal 3 by the user based on the visualization data provided to the user based on the visualization data provided by the visualization data providing unit 161. The specified location information can be obtained from the user terminal.

As described above, FIG. 1 illustrates a case in which the autonomous behavior robot 1 is configured such that the data providing device 10 and the robot 2 are separated from each other. May be included. For example, the robot 2 may include all functions of the data providing device 10. The data providing device 10 may be a device that temporarily substitutes a function when the processing capability of the robot 2 is insufficient, for example.

Also, in the present embodiment, “acquisition” may mean that the subject to be acquired actively acquires, or the subject to acquire may passively acquire. For example, the designation acquisition unit 162 may acquire the space data by receiving an instruction for creating spatial data transmitted from the user terminal 3 by the user, or store the data in a storage area (not shown) not shown by the user. The instruction for creating the created spatial data may be obtained by reading from the storage area.

In addition, the first communication control unit 11, the point cloud data generation unit 12, the spatial data generation unit 13, the visualization data generation unit 14, the imaging target recognition unit 15, the second communication control unit 16, the captured image The functional units of the acquiring unit 111, the spatial data providing unit 112, the instruction unit 113, the visualization data providing unit 161, the designation acquiring unit 162, and the execution information acquiring unit 163 are examples of the function of the autonomous behavior robot 1 in the present embodiment. It is shown, and does not limit the functions of the autonomous behavior robot 1. For example, the autonomous behavior robot 1 does not need to have all the functional units included in the data providing device 10 and may have some functional units. In addition, the autonomous behavior robot 1 may have other functional units other than those described above. Further, each function unit of the marker recognition unit 22, the movement control unit 23, the state information acquisition unit 24, the search unit 25, and the notification operation execution unit 26 of the robot 2 has the functions of the autonomous behavior robot 1 in the present embodiment. This is an example, and does not limit the functions of the autonomous robot 1. For example, the autonomous behavior robot 1 does not need to have all the functional units of the robot 2 but may have some of the functional units.

The above-described functional units of the autonomous behavior robot 1 have been described as being realized by software as described above. However, at least one or more of the above functions of the autonomous behavior robot 1 may be realized by hardware.

In addition, any of the above functions of the autonomous behavior robot 1 may be implemented by dividing one function into a plurality of functions. Further, any two or more of the functions of the autonomous behavior robot 1 may be integrated into one function. That is, FIG. 1 illustrates the functions of the autonomous behavior robot 1 by functional blocks, and does not indicate that each function is configured by a separate program file, for example.

The autonomous behavior robot 1 may be a device realized by one housing or a system realized by a plurality of devices connected via a network or the like. For example, the autonomous behavior robot 1 may realize some or all of its functions by a virtual device such as a cloud service provided by a cloud computing system. That is, the autonomous behavior robot 1 may realize at least one or more of the above functions in another device. In addition, the autonomous behavior robot 1 may be a general-purpose computer such as a tablet PC, or may be a dedicated device having limited functions.

The autonomous behavior robot 1 may realize some or all of its functions in the robot 2 or the user terminal 3.

Next, the hardware configuration of the autonomous behavior robot 1 (control unit of the robot 2) will be described with reference to FIG. FIG. 2 is a block diagram illustrating an example of a hardware configuration of the autonomous behavior robot 1 according to the embodiment.

The autonomous behavior robot 1 has a CPU (Central Processing Unit) 101, a RAM (Random Access Memory) 102, a ROM (Read Only Memory) 103, a touch panel 104, a communication I / F (Interface) 105, a sensor 106, and a clock 107. . The autonomous behavior robot 1 is a device that executes the autonomous behavior robot control program described with reference to FIG.

(4) The CPU 101 controls the autonomous behavior robot 1 by executing the autonomous behavior robot control program stored in the RAM 102 or the ROM 103. The autonomous behavior type robot control program is acquired from, for example, a recording medium on which the autonomous behavior type robot control program is recorded, or a program distribution server via a network, installed in the ROM 103, read from the CPU 101, and executed.

(4) The touch panel 104 has an operation input function and a display function (operation display function). The touch panel 104 enables a user of the autonomous behavior robot 1 to perform an operation input using a fingertip, a touch pen, or the like. A case will be described in which the autonomous behavior robot 1 according to the present embodiment uses the touch panel 104 having an operation display function. However, the autonomous behavior robot 1 has a display device having a display function and an operation input device having an operation input function separately. You may have. In that case, the display screen of the touch panel 104 can be implemented as the display screen of the display device, and the operation of the touch panel 104 can be implemented as the operation of the operation input device. Note that the touch panel 104 may be realized by various forms such as a head-mounted type, glasses type, and wristwatch type displays.

The communication I / F 105 is a communication I / F. The communication I / F 105 executes, for example, short-range wireless communication such as a wireless LAN, a wired LAN, and infrared rays. Although only the communication I / F 105 is shown in FIG. 2 as the communication I / F, the autonomous behavior robot 1 may have each communication I / F in a plurality of communication methods. The communication I / F 105 may perform communication with a control unit that controls the imaging unit 21 (not shown) or a control unit that controls the moving mechanism 29.

The sensor 106 is hardware such as a camera of the photographing unit 21, a TOF or a thermo camera, and hardware such as a microphone, a thermometer, an illuminometer, or a proximity sensor. Data obtained by these hardware is stored in the RAM 102 and processed by the CPU 101.

The clock 107 is an internal clock for acquiring time information. The time information acquired by the clock 107 is used, for example, to confirm the time at which the notification operation is performed. The microphone 108 collects surrounding sounds. The microphone 108 collects, for example, the voice of the notification target person.

The speaker 109a, the display 109b, and the actuator 109c are specific hardware examples of the notification unit 27 described with reference to FIG. The speaker 109a outputs sound, the display 109b outputs display data, and the actuator 109c is a movable unit. The notification unit 27 may have hardware other than the speaker 109a, the display 109b, and the actuator 109c.

Next, the operation of providing the visualization data of the robot control program will be described with reference to FIG. FIG. 3 is a flowchart illustrating a first example of the operation of the robot control program according to the embodiment. In the following description of the flowchart, it is assumed that the execution subject of the operation is the autonomous robot 1, but each operation is executed in each function of the autonomous robot 1 described above.

In FIG. 3, the autonomous behavior robot 1 determines whether or not a captured image has been acquired (step S11). The determination as to whether or not a captured image has been acquired can be made based on whether or not the captured image acquisition unit 111 has acquired a captured image from the robot 2. The determination as to whether or not a captured image has been obtained is made in units of processing of the captured image. For example, when the captured image is a moving image, the moving image is continuously transmitted from the robot 2. Therefore, the determination as to whether or not the captured image has been obtained depends on whether the number of frames or the data amount of the obtained moving image is a predetermined value. Can be done depending on whether or not The captured image may be acquired mainly by the mobile robot and transmitting the captured image, or may be acquired by the captured image acquisition unit 111 and taking the captured image from the mobile robot. If it is determined that the photographed image has not been acquired (step S11: NO), the autonomous behavior robot 1 repeats the process of step S11 and waits for the photographed image to be acquired.

On the other hand, if it is determined that the captured image has been acquired (step S11: YES), the autonomous robot 1 generates point cloud data (step S12). The point cloud data is generated by the point cloud data generation unit 12 detecting, for example, a point having a large change in luminance in a captured image as a feature point, and giving three-dimensional coordinates to the detected feature point. Can be performed. The feature point may be detected, for example, by performing a differentiation process on the captured image, detecting a change in the gradation of the captured image, and detecting a portion having a large change in the gradation. The assignment of the coordinates to the feature points may be executed by detecting the same feature point photographed from different photographing angles. The determination as to whether or not a captured image has been obtained in step S11 can be made based on whether or not captured images captured from a plurality of directions have been obtained.

After executing the process of step S12, the autonomous behavior robot 1 generates spatial data and recognizes a marker (step S13). The generation of the spatial data can be executed by the spatial data generating unit 13 performing, for example, Hough transform of the point cloud data. The details of step S13 will be described with reference to FIG.

(4) After executing the processing in step S13, the autonomous behavior robot 1 provides the generated spatial data to the robot 2 (step S14). As shown in FIG. 3, the spatial data may be provided to the robot 2 every time the spatial data is generated, or may be provided asynchronously with the processing shown in steps S11 to S18. Good. The robot 2 provided with the spatial data can grasp the movable range based on the spatial data.

After executing the process of step S14, the autonomous behavior robot 1 determines whether or not to recognize a space element (step S15). The determination as to whether or not to recognize a spatial element can be performed by, for example, setting whether to recognize a spatial element in the imaging target recognition unit 15. Even if it is determined that the space element is recognized, if the recognition fails, it may be determined that the space element is not recognized.

場合 If it is determined that the space element is recognized (step S15: YES), the autonomous behavior robot 1 generates first visualization data (step S16). The generation of the first visualization data can be executed by the visualization data generation unit 14. The first visualization data is visualization data generated after the imaging target recognition unit 15 recognizes a spatial element. For example, when the imaging target recognizing unit 15 determines that the spatial element is a table, the visualization data generation unit 14 determines whether the spatial element is a table even if the upper surface of the table is not imaged and has no point cloud data. The visualization data can be generated as if the top surface were flat. Further, when it is determined that the space element is a wall, the visualization data generation unit 14 can generate the visualization data assuming that a part that is not photographed is also a plane.

場合 If it is determined that the space element is not recognized (step S15: NO), the autonomous behavior robot 1 generates the second visualization data (step S17). The generation of the second visualization data can be executed by the visualization data generation unit 14. The second visualization data is visualization data generated by the imaging target recognition unit 15 without recognizing a spatial element, that is, based on point cloud data and spatial data generated from a captured image. The autonomous behavior robot 1 can reduce the processing load by not performing the recognition processing of the space element.

(4) After executing the processing of step S16 or the processing of step S17, the autonomous behavior robot 1 provides visualization data (step S18). The visualization data is provided by the visualization data providing unit 161 providing the visualization data generated by the visualization data generation unit 14 to the user terminal 3. The autonomous behavior robot 1 may generate and provide visualization data in response to a request from the user terminal 3, for example. After executing the processing of step S18, the autonomous behavior robot 1 ends the operation shown in the flowchart.

Next, an operation related to generation of spatial data of the robot control program will be described with reference to FIG. FIG. 4 is a flowchart illustrating a second example of the operation of the robot control program according to the embodiment.

において In FIG. 4, the autonomous behavior robot 1 generates spatial data (step S131). The generation of the spatial data can be executed by the spatial data generating unit 13 performing, for example, Hough transform of the point cloud data. After executing step S131, the autonomous behavior robot 1 determines whether or not the marker has been recognized (step S132). Whether or not the marker has been recognized can be determined based on whether or not the marker recognizing unit 22 has recognized the image of the marker in the image captured by the image capturing unit 21. The robot 2 can notify the data providing device 10 of the marker recognition result.

If it is determined that the marker has been recognized (step S132: YES), the autonomous behavior robot 1 sets a limited range in which movement is limited to the spatial data generated in step S121 (step S133).

後 After executing the processing of step S133, or when determining that the marker has not been recognized (step S132: NO), the autonomous behavior robot 1 ends the operation of generating the provided data of step S13 shown in the flowchart.

Next, the notification operation of the robot control program will be described with reference to FIG. FIG. 5 is a flowchart when the autonomous behavior robot control program according to the embodiment executes a wake-up operation as a notification operation.

In FIG. 5, the autonomous behavior robot 1 determines whether or not the execution information has been acquired (step S21). The determination as to whether the execution information has been acquired can be made based on whether the execution information acquisition unit 163 has acquired the execution information from the user terminal 3. If it is determined that the execution information has not been acquired (step S21: NO), the autonomous behavior robot 1 repeats the process of step S21 and waits for the acquisition of the execution information.

On the other hand, if it is determined that the execution information has been obtained (step S21: YES), the autonomous behavior robot 1 calculates a movement route (step S22). The calculation of the moving route can be executed by the search unit 25 calculating based on the location information included in the execution information.

After executing the processing of step S22, the autonomous behavior robot 1 starts searching for a notification target person and starts moving along the calculated moving route (step S23). The search for the notification target person can be executed by the search unit 25. The movement can be executed by controlling the movement mechanism 29 by the movement control unit 23. The start of the search in step S23 is executed based on the time information, and the autonomous robot 1 starts moving before the designated execution time. For example, when the execution time is set at 6:00 am, the autonomous behavior robot 1 considers the traveling time on the traveling route and takes the robot 2 at a time before 6:00 am so that the notification operation can be performed at the execution time. Start moving. The time at which the movement is started may be automatically determined by the autonomous robot 1 based on the time information, or may be manually set by the user.

After performing the process of step S23, the autonomous behavior robot 1 determines whether the robot 2 has reached the location specified as the search location (step S24). Whether or not the search location has been reached can be determined, for example, by the movement control unit 23 comparing the search location with the current position (coordinate position) of the robot 2. If it is determined that the robot has not reached the search location (step S24: NO), the autonomous robot 1 repeats the process of step S24, and waits for the robot 2 to reach the search location.

On the other hand, if it is determined that the robot has arrived at the search place (step S24: YES), the autonomous robot 1 determines whether or not the notification target person has been found (step S25). Whether the notification target person has been found or not can be determined by, for example, the search unit 25 based on the notification target person information included in the execution information. Note that whether or not the notification target person has been found may be determined, for example, when a person is present at the search location, by regarding that person as the notification target person. When it is determined that the notification target person has not been found (step S25: NO), the autonomous behavior robot 1 returns to the process of step S23 and starts moving to the next search location. That is, the autonomous behavior robot 1 can sequentially search for a plurality of designated search locations and find a notification target person. If the notification target person cannot be found at the designated search location, the search operation in the illustrated flowchart is terminated, and the fact that the notification target person was not found is recorded or the user terminal 3 is notified. It may be.

On the other hand, if it is determined that the notification target person has been found (step S25: YES), the autonomous behavior robot 1 acquires state information (step S26). The acquisition of the state information can be executed by the state information acquisition unit 24. In the illustrated flowchart, it is assumed that information as to whether the notification target person is sleeping or waking up is acquired as state information.

After executing the process of step S26, the autonomous behavior robot 1 determines whether the notification target person is sleeping (step S27). Whether or not the notification target person is sleeping can be determined by the state information acquisition unit 24. When it is determined that the notification target person is sleeping (step S27: YES), the autonomous behavior robot 1 executes a wake-up operation as a notification operation (step S28). The execution of the wake-up operation can be executed by the notification operation execution unit 26. The notification operation execution unit 26 executes the wake-up operation when the current time reaches a predetermined execution time. That is, when the autonomous behavior robot 1 arrives at the search place before the execution time, it waits until the execution time comes. In addition, when the robot arrives at the search place after the execution time, the autonomous behavior robot 1 immediately executes the wake-up operation. The wake-up operation may be performed when the sleep state of the notification target person is determined by the state information acquisition unit 24 and the notification target person is in the predetermined sleep state, for example.

実行 After executing the processing of step S28, the autonomous behavior robot 1 executes the processing of step S27 again to determine whether or not the notification target person is sleeping. The re-execution of the process in step S27 may be executed after a certain time elapses, for example, 5 minutes, 10 minutes, or 15 minutes.

On the other hand, when it is determined that the notification target person is not sleeping (was awake) (step S27: NO), the autonomous behavior robot 1 performs a greeting operation (step S29), and performs the operation shown in the flowchart. finish. The greeting operation is, for example, a fixed phrase such as "Good morning" or an audio output such as the current time. Whether to perform the greeting operation may be set in advance in the execution information.

The processing in each step in the operation (robot control method) of the robot control program described in the present embodiment does not limit the execution order.

Next, the execution information will be described with reference to FIG. FIG. 6 is a diagram illustrating an example of execution information according to the embodiment.

実行 In FIG. 6, execution information 1000 has data items of “notification target person” and “notification operation”. The execution information 1000 is set in the user terminal 3 and can be acquired by the execution information acquisition unit 163.

"Notification target person" is information for specifying a notification target person who performs a notification operation. In the figure, “A”, “B”, and “C” are specified in the ID of the “notification target”, but the number of notification targets is an arbitrary number of one or more. In addition, the physical characteristics for specifying each notification target person may be registered in the autonomous behavior robot 1 in advance.

"Notification operation" is information on the notification operation to be performed on the notification target person. A plurality of notification operations can be set for one notification target person. The figure shows that two notification operations of “notification operation 1” and “notification operation 2” are set for each notification target person. A priority may be set for each notification operation for each notification target person.

“Notification operation” has data items of “search location”, “notification operation”, and “time”. “Search location” is information indicating a search location for searching for a notification target person. The figure shows a case where a room such as “child room”, “bedroom”, “western room”, or “living room” is designated as the search place. Information on each room is provided to the user terminal 3 from the visualization data providing unit 161 and can be shared between the autonomous robot 1 and the user terminal 3. The user can specify a search location by selecting a room from the map displayed on the user terminal 3.

"Notification operation" is information indicating a notification operation to be performed on a notification target person. In the “notification operation”, a notification operation such as “alarm operation” or “broadcast time notification” is set. A responsibility level can be set for each notification operation. The responsibility level is information indicating the importance of the notification operation performed by the autonomous behavior robot 1 on the notification target person. The autonomous behavior robot 1 can change the notification operation according to the set responsibility level. For example, the autonomous behavior robot 1 performs, in accordance with the responsibility level, the execution order (priority) of the notification operation, the magnitude of the sound output in the notification operation, the content of the sound, the number of times the notification operation is executed, or the end condition of the notification operation. Etc. may be changed. For example, when performing a plurality of notification operations, the autonomous behavior robot 1 preferentially executes a notification operation having a high responsibility level. In addition, the autonomous behavior robot 1 can make the notification target person recognize the notification operation with a high probability by increasing the voice output or increasing the number of executions in the notification operation having a high responsibility level. The figure illustrates a case where three levels of “high level”, “medium level” and “low level” are set as the responsibility level. The autonomous behavior robot 1 executes the “high level” notification operation prior to the “medium level” notification operation, and further executes the “medium level” notification operation prior to the “low level” notification operation. You may.

“Alarm operation” can be set to “with greeting” or “without greeting”. In FIG. 5, the operation in the case of “with greeting” has been described. “Broadcast time notification” is an operation of notifying the start of a television broadcast or the like. “Outing time notification” is an operation of notifying the notification target person of the leaving time.

“Time” is time information for executing the notification operation. In the “time”, a time for the notification operation executed only once is set. In addition, the time for the notification operation that is repeated every day may be set in “time”.

Although the figure shows two notification operations as one piece of execution information for each of three notification targets, for example, the execution information acquisition unit 163 acquires each piece of execution information individually and merges the pieces of execution information. By doing so, the notification operation may be scheduled in consideration of the priority order of the notification operations in a plurality of pieces of execution information, the possibility of execution due to interference in execution time, and the like.

Next, a method for setting execution information will be described with reference to FIG. FIG. 7 is a diagram illustrating an example of a method of setting execution information according to the embodiment.

において In FIG. 7, an execution information setting screen 30 is displayed on the display screen of the user terminal 3. The execution information setting screen 30 includes a notification target person setting unit 311, a time setting unit 312, a notification operation setting unit 313, and a search place setting unit 32. The execution information setting screen 30 is displayed in, for example, an application program (app) of the user terminal 3.

The notification target person setting unit 311 is a pull-down menu for selecting a notification target person. The figure shows that the notification target person A is selected as the notification target person. The time setting unit 312 is a pull-down menu for selecting time information for executing the notification operation. The notification operation setting section 313 is a pull-down menu for selecting a notification operation. The figure shows that the wake-up operation is selected as the notification operation.

The search place setting unit 32 displays, for example, a plan view of the arrangement of the rooms at home based on the visualization data provided from the visualization data providing unit 161 and sets a search place to search for a notification target person from the plan view. Is displayed. The figure shows that the search position setting unit 32 displays a home position 321, a child room 322, a bedroom 323, and a Western room 324 to which the robot 2 returns for charging. The user sets the selected place by touching at least one of the children's room 322, the bedroom 323, and the Western room 324 in the search place setting unit 32.

For example, when the user sets the child room 322 as a search place, the search place setting unit 32 displays the moving route 325. The movement route 325 is calculated by the search unit 25 and provided to the user terminal 3, for example. The figure shows that the moving route 325 from the home position 321 to the child room 322 is indicated by a broken line. In addition, the search location setting unit 32 may display that there is a problem with the movement when the movement is restricted on the movement route. For example, when there is a step at the entrance of the child room 322 and the robot 2 cannot move, the fact may be displayed on the movement route 325 with an X mark.

Note that a program for realizing the functions constituting the apparatus described in the present embodiment is recorded on a computer-readable recording medium, and the program recorded on the recording medium is read and executed by a computer system. Thus, the various processes described above in the present embodiment may be performed. Here, the “computer system” may include an OS and hardware such as peripheral devices. The “computer system” also includes a homepage providing environment (or a display environment) if a WWW system is used. The “computer-readable recording medium” includes a writable nonvolatile memory such as a flexible disk, a magneto-optical disk, a ROM, and a flash memory, a portable medium such as a CD-ROM, and a hard disk incorporated in a computer system. Storage device.

Further, a “computer-readable recording medium” refers to a volatile memory (for example, a DRAM (Dynamic)) in a computer system that becomes a server or a client when a program is transmitted via a network such as the Internet or a communication line such as a telephone line. Random (Access @ Memory)), which includes a program that is held for a certain period of time. Further, the above program may be transmitted from a computer system storing the program in a storage device or the like to another computer system via a transmission medium or by a transmission wave in the transmission medium. Here, the "transmission medium" for transmitting a program refers to a medium having a function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line. Further, the program may be for realizing a part of the functions described above. Furthermore, what realizes the above-described functions in combination with a program already recorded in the computer system, a so-called difference file (difference program) may be used.

The embodiment of the present invention has been described with reference to the drawings. However, the specific configuration is not limited to this embodiment, and various modifications (modifications) may be made without departing from the spirit of the present invention. Example) is also included.

As a modified example, for example, a location where the user terminal 3 exists may be set as a search location, or a search operation may be performed as the search location. The location where the user terminal 3 is located can be grasped by, for example, the positional relationship between a base station for wireless LAN communication (not shown) and the user terminal 3. Further, the location where the user terminal 3 exists can be grasped by the radio field intensity of the short-range wireless communication between the robot 2 and the user terminal 3. By searching for a notification target person based on the location of the user terminal 3, the user can set the search location simply by bringing the user terminal to the search location, thereby simplifying the setting of the notification operation. It becomes possible.

Also, the application of the user terminal 3 may call the robot 2 at a predetermined time at which the notification operation is performed, and cause the notification operation execution unit 26 to execute the notification operation. For example, when the user sets a timer in the wake-up function of the user terminal 3, the application of the user terminal 3 may call the robot 2 at the set time of the timer to execute the notification operation. That is, the application of the user terminal 3 can link the general alarm function in the user terminal 3 with the robot 2. For example, the wake-up function includes a "snooze" function in which, once the alarm is stopped, the alarm sounds again at a predetermined number of repetitions after a predetermined time has elapsed. The application of the user terminal 3 may call the robot 2 when the alarm is started by the snooze function and execute the notification operation. Further, the application of the user terminal 3 may call the robot 2 and execute the notification operation when the repetition number of the snooze ends.

The setting of the execution information may be performed by setting a marker. For example, a marker indicating a search place or a person to be notified may be set at the entrance of the room, so that the room where the marker is installed may be set as the search place, or the person to be notified may be set. For example, the person who wants the notification operation may set the marker indicating that the notification operation is desired on the door knob of the room, so that the robot 2 recognizes the marker and receives the notification operation by the robot 2. In addition, a person who does not desire the notification operation may avoid the notification operation by the robot 2 by setting a marker (such as “Don't Disturb” display) indicating that the notification operation is not desired on the door knob of the room. Good.

(4) The setting of the execution information may be performed by the implemented notification operation. For example, the execution information may be set by notifying the robot 2 that the notification target person has executed the notification operation, such as “wake up at the same time tomorrow”. By setting the execution information by the performed notification operation, the setting of the execution information can be facilitated.

In addition, when the robot 2 detects that the notification target person has performed an unpleasant act via a sensor or the like, the robot 2 may execute a motion that dislikes the execution of the next notification operation. An unpleasant act is an act different from a pleasant act, for example, an act of hitting the robot 2 or scolding the robot 2. When the robot 2 detects through a sensor or the like that the notification target person has performed an action such as apologizing to the robot 2 performing a motion that dislikes performing the notification operation, the robot 2 corrects the mood and notifies the robot 2 A motion that undertakes the execution of the operation may be executed. As described above, by reacting to the uncomfortable act, it is possible to deepen the communication with the robot 2.

Further, it is assumed that the robot 2 cannot move to the inside of the room because the door of the room where the notification target person is located is closed. In this case, the robot 2 may stop in front of the door and execute the notification operation when the execution time comes. The notifying operation execution unit 26 is not limited to the case where the door is closed, and stays there when it is not possible to reach the search location due to an obstacle or a no-entry area. May be executed. The notification operation execution unit 26 may execute the notification operation at a volume higher than the volume emitted when the search location can be reached. That is, the notification operation execution unit 26 performs the notification operation at a volume higher than usual so that the notification target person notices, considering that the user is away from the location specified as the search location. As the notification action, not only the sound is generated but also the sound may be generated by lightly hitting the body of the robot 2 against a door or a wall. In addition, the robot 2 may perform the notification operation at a position that does not collide with a door or a person by opening the door. Therefore, the notification operation execution unit 26 recognizes the open / close range of the door. Then, the notification operation execution unit 26 moves the robot 2 outside the range, and executes the notification operation when the robot 2 is located outside the opening / closing range of the door. Thereby, when the notification target person notices the notification and opens the door, the contact between the robot 2 and the door can be prevented. Also, the search unit 25 breaks the entry prohibition rule specified by the marker, moves through the entry prohibition range, and moves to the search place where the notification target person is located only when moving for the notification operation. You may.

<Acquisition of execution information by voice input>
The autonomous behavior robot 1 may acquire the execution information by voice input. An outline of a process of acquiring execution information by voice input will be described.

FIG. 8 is a diagram showing an example of a module configuration of the autonomous behavior robot 1 for acquiring execution information by voice input. The autonomous behavior robot 1 includes a microphone 108, a voice recognition unit 201, an execution information acquisition unit 163, and an execution information storage unit 204. The autonomous behavior robot 1 is configured as a system including the robot 2 and the data providing device 10 as described above. FIG. 8 shows extracted functions related to “acquisition of execution information by voice input”. It is determined whether the functions of the microphone 108, the voice recognition unit 201, the execution information storage unit 204, and the execution information acquisition unit 163 are realized by the robot 2, the data providing device 10, or another device. It may be arbitrarily designed based on the first specification.

The microphone 108 inputs the voice of the user. The microphone 108 may be provided in either the robot 2 or the data providing device 10. Moreover, the microphone 108 may be provided outside the autonomous behavior robot 1. For example, a microphone installed indoors where the robot 2 acts may be used. Alternatively, the microphone of the user terminal 3 may be used. When the execution information is input by voice, the user utters a linguistic expression such as a phrase or a sentence for specifying the execution information toward the microphone. In this example, it is assumed that the user who is a wife utters a command "Wake up the dad at bed time at 7 o'clock" in order to wake up the user who is a husband. The voice recognition unit 201 recognizes voice data input by the microphone 108 and converts the voice data into language data.

The execution information acquisition unit 163 has an execution information identification unit 202 and a conversion rule storage unit 203. The execution information specifying unit 202 specifies the execution information based on the language data by referring to the conversion rules stored in the conversion rule storage unit 203. The conversion rule associates a parameter of execution information with a language expression. The conversion rule storage unit 203 specifically stores an operation expression corresponding to the type of notification operation, a person expression corresponding to the notification target person ID, and a location expression corresponding to an indoor place such as a child room or a bedroom. I have. For example, the motion expression “wake up” is associated with the notification operation “alarm / greeting”, the person expression “dad” is associated with the notification target ID “A”, and the place “bedroom” is “bed”. A location expression is associated. The conversion rule may associate a time expression corresponding to the time. For example, a time expression “lunch” may be associated with time “12:00”.

When the language data includes an action expression corresponding to the type of the notification operation, the execution information specifying unit 202 specifies the type of the notification operation corresponding to the operation expression. Further, when the language data includes a person expression corresponding to the notification target person ID, the execution information specifying unit 202 specifies the notification target person ID corresponding to the person expression. Furthermore, when the language data includes a place expression corresponding to an indoor place, the execution information specifying unit 202 specifies an indoor place corresponding to the place expression. The specified indoor place corresponds to a search place. The execution information specifying unit 202 specifies the time when the language data includes a time expression such as “7 o'clock”, and specifies the time when the time expression corresponding to the time is included. Specifies the time corresponding to the time expression. The specified time corresponds to the notification time. The execution information specifying unit 202 specifies execution information including a notification target person ID, a search place, a notification operation type, and a notification time. In this example, the execution information including the notification target person ID “A”, the search place “bedroom”, the notification operation type “alarm / greeting”, and the notification time “7:00” is specified. The execution information storage unit 204 stores the specified execution information.

実装 Hereinafter, eight implementation examples in which the autonomous behavior robot 1 acquires execution information by voice input will be described. In the first implementation example, the functions of all the modules shown in FIG. That is, the user's voice is input through the microphone 108 provided in the data providing apparatus 10, and the data providing apparatus 10 performs a process of voice recognition and a process of specifying execution information.

The data providing apparatus 10 includes the microphone 108, the voice recognition unit 201, the execution information specifying unit 202, the conversion rule storage unit 203, and the execution information storage unit 204. The voice recognition unit 201 of the data providing device 10 receives voice data from the microphone 108 of the data providing device 10 and converts the voice data into language data. The execution information specifying unit 202 of the data providing apparatus 10 specifies execution information based on language data with reference to the conversion rule storage unit 203 of the data providing apparatus 10. The execution information storage unit 204 of the data providing device 10 stores the specified execution information.

When the location information is not included in the execution information specified in the first implementation example, the execution information specifying unit 202 determines the location of the microphone 108 that has input the voice, that is, the location where the data providing apparatus 10 is installed. The location information in the execution information may be used.

In the second implementation example, the function of the module shown in FIG. 8 is realized by the attachment type device 4 and the data providing device 10 attached indoors where the robot 2 acts. That is, the user's voice is input through the microphone 108 included in the attached device 4, and the data providing device 10 performs the process of voice recognition and the process of specifying execution information. For this purpose, audio data is transmitted from the attached device 4 to the data providing device 10. The one or more attachable devices 4 are identified by IDs, and the data providing apparatus 10 pre-registers the installation location (indoor position coordinates) of each attachable device 4.

A plurality of attachment type devices 4 may be attached to different places. The attached device 4 can transmit data to and from the data providing device 10 by one or both of wired communication and wireless communication. The method of wireless communication may be, for example, short-range wireless communication such as wireless LAN, Bluetooth (registered trademark), or infrared communication, or wired communication. The wired communication system may be, for example, a wired LAN.

The attachment type device 4 has a microphone 108 and an audio data transmission unit (not shown). The audio data transmission unit transmits the audio data input by the microphone 108 of the attachment type device 4 to the data providing device 10. The data providing device 10 includes a voice data receiving unit (not shown), a voice recognition unit 201, an execution information specifying unit 202, a conversion rule storage unit 203, and an execution information storage unit 204. The audio data receiving unit receives the audio data sent from the attached device 4. The voice recognition unit 201 of the data providing device 10 converts the voice data received by the voice data receiving unit into language data. The execution information specifying unit 202 of the data providing device 10 specifies the execution information based on the language data with reference to the conversion rule storage unit 203. 204 of the data providing apparatus 10 stores the specified execution information.

When the location information is not included in the execution information specified in the second implementation example, the execution information acquisition unit 163 determines the location of the microphone 108 that has input the voice, that is, the location where the attachable device 4 is attached. The location information in the execution information may be used. For example, when a voice saying “Please wake up after 30 minutes” is input from the microphone 108 attached to the bedroom, the bedroom may be used as the location information in the execution information. This example sentence assumes that the user is alone and the notification target person ID can be omitted. Further, the execution information specifying unit 202 specifies the time to be notified by adding 30 minutes to the current time based on the time expression “after 30 minutes”. Further, it is assumed that the execution information specifying unit 202 specifies the type of the notification operation “no alarm / no greeting” by the operation expression “Please wake up”.

FIG. 9 is a diagram illustrating an example of a module configuration of the data providing apparatus 10 relating to the specification of the mounting location of the mounting type device 4. In other words, FIG. 9 shows a module configuration example of the data providing apparatus 10 corresponding to the second implementation example, particularly, the location specifying function of the attached device 4 (microphone 108) that picked up the voice. The execution information acquisition unit 163 of the data providing device 10 includes a location identification unit 211 and an attachment location storage unit 212. The attachment location storage unit 212 stores the attachment location of each attachment type device 4 attached to a predetermined indoor location. That is, the attachment location storage unit 212 stores the attachment location in association with the ID of each attachment type device 4. In the second implementation example, after the execution information is specified by the execution information specifying unit 202, the location specifying unit 211 determines whether or not the execution information includes the location information. When the location information is included in the execution information, the location specifying unit 211 ends the processing as it is. If the location information is not included in the execution information, the location specifying unit 211 obtains the ID of the attached device 4 that is the transmission source of the audio data from the audio data receiving unit. Then, the location specifying unit 211 refers to the mounting location storage unit 212 to specify a mounting location corresponding to the ID of the source mountable device 4, and stores the specified mounting location as a search location in the execution information storage unit 204. Write. That is, the attachment location of the attachment type device 4 is stored as the location information in the execution information.

In the third implementation example, the functions of the module shown in FIG. 8 are realized by the user terminal 3 and the data providing device 10. The user terminal 3 here may be any computer terminal such as a smartphone or a laptop PC. In the third implementation example, the microphone 108 built in the user terminal 3 is used. The user's voice is input through the microphone 108 of the user terminal 3, and the data providing device 10 performs the process of voice recognition and the process of specifying execution information. For that purpose, voice data is transmitted from the user terminal 3 to the data providing device 10.

The user terminal 3 in the fourth implementation example has the microphone 108 and a voice data transmission unit (not shown). The audio data transmission unit transmits the audio data input by the microphone 108 of the user terminal 3 to the data providing device 10. The module configuration of the data providing device 10 is the same as in the second implementation example. That is, the third implementation example is different from the second implementation example in that the user terminal 3 is used as a movable general-purpose product, instead of the fixed type and the attached device 4 as a dedicated product.

In the fourth implementation example, the user terminal 3 and the data providing device 10 also realize the functions of the module shown in FIG. A user's voice is input to the user terminal 3 and a voice recognition process is performed. The data providing device 10 performs a process of specifying execution information. For this purpose, language data is transmitted from the user terminal 3 to the data providing device 10. The fourth implementation example differs from the third implementation example in that the user terminal 3 performs not only speech acquisition but also speech recognition.

The user terminal 3 in the fourth implementation example has the microphone 108, the voice recognition unit 201, and the language data transmission unit (not shown). The voice recognition unit 201 of the user terminal 3 converts voice data input by the microphone 108 of the user terminal 3 into language data. The language data transmitting unit transmits the converted language data to the data providing device 10. The data providing device 10 includes a language data receiving unit (not shown), an execution information specifying unit 202, a conversion rule storage unit 203, and an execution information storage unit 204. The language data receiving unit receives the language data sent from the user terminal 3. The execution information specifying unit 202 of the data providing device 10 refers to the conversion rule storage unit 203 of the data providing device 10 and specifies execution information based on the received language data. The execution information storage unit 204 of the data providing device 10 stores the specified execution information.

In the fifth implementation example, the user terminal 3 and the data providing device 10 also realize the functions of the module shown in FIG. A user's voice is input to the user terminal 3, and a process of voice recognition and a process of specifying execution information are performed. Then, the execution information is transmitted from the user terminal 3 to the data providing device 10. The fifth implementation example is different from the fourth implementation example in that the user terminal 3 also specifies execution information in addition to voice acquisition and voice recognition.

The user terminal 3 in the fifth implementation example includes the microphone 108, the voice recognition unit 201, the execution information specifying unit 202, the conversion rule storage unit 203, and the execution information transmission unit (not shown). The voice recognition unit 201 of the user terminal 3 converts voice data input by the microphone 108 of the user terminal 3 into language data. The execution information specifying unit 202 refers to the conversion rule storage unit 203 and specifies execution information based on the converted language data. The execution information transmitting unit transmits the specified execution information to the data providing device 10. The data providing device 10 includes an execution information receiving unit (not shown) and an execution information storage unit 204. The execution information receiving unit receives the execution information sent from the user terminal 3. The execution information storage unit 204 of the data providing device 10 stores the received execution information.

In the third to fifth implementation examples, when the execution information acquired by voice input does not include the location information, the execution information acquisition unit 163 determines the location of the microphone 108 that has input the voice, that is, the user terminal. The location 3 may be used as the location information in the execution information. As described above, the third to fifth implementation examples are common in that the processing is started with the voice data acquired by the user terminal 3 as a starting point.

FIG. 10 is a diagram illustrating an example of a module configuration of the data providing apparatus 10 relating to the specification of the location of the user terminal 3. In other words, FIG. 10 illustrates an example of a module configuration related to the user terminal 3 and the data providing device 10 corresponding to the third to sixth implementation examples, and in particular, the location configuration function of the user terminal 3 (the movable microphone 108). Show. The user terminal 3 includes a position measuring unit 221 and a terminal position transmitting unit 222. The position measuring unit 221 measures the current position of the user terminal 3 based on, for example, beacon signals transmitted from a plurality of beacon transmitters installed at predetermined indoor locations. In the case of this method, the user terminal 3 includes a beacon receiver, receives a beacon signal transmitted by a beacon transmitter installed at a predetermined position, and specifies the ID of the beacon transmitter. The position measuring unit 221 specifies the current position of the user terminal 3 by analyzing the relationship between the radio wave intensity of the received beacon signal and the interval between the beacon transmitter specified by the ID and the user terminal 3. The beacon transmitter may be included in the attached device 4 or may be provided separately from the attached device 4.

The terminal position transmitting unit 222 transmits the current position of the user terminal 3 to the data providing device 10. In the case of the third implementation example, the terminal position transmission unit 222 transmits the current position of the user terminal 3 before or after transmission of audio data in the audio data transmission unit, for example. In the case of the fourth implementation example, the terminal position transmitting unit 222 transmits, for example, the current position of the user terminal 3 before or after the transmission of the language data in the language data transmitting unit. In the case of the fifth implementation example, the terminal position transmission unit 222 transmits the current position of the user terminal 3 before or after transmission of the execution information in the execution information transmission unit, for example.

The data providing device 10 includes a terminal position receiving unit 223, a location specifying unit 224, a floor plan data storage unit 225, and an execution information storage unit 204. The location specifying unit 224 and the floor plan data storage unit 225 may be included in the execution information acquisition unit 163. The floor plan data storage unit 225 stores a range of a place for each indoor place. The place referred to here is an indoor area such as a children's room or a bedroom. The terminal position receiving unit 223 receives the current position of the user terminal 3 from the user terminal 3. After the execution information is specified by the execution information specifying unit 202, the location specifying unit 224 determines whether or not the execution information includes the location information. When the location information is included in the execution information, the location specifying unit 224 ends the processing. If the location information is not included in the execution information, the location specifying unit 224 obtains the current location of the user terminal 3 from the terminal location receiving unit 223. Then, the location specifying unit 224 refers to the floor plan data storage unit 225 and writes the location including the current position of the user terminal 3 as the location information in the execution information stored in the execution information storage unit 204. Since the location information in the execution information indicates the search location, it means that the search is performed at a location including the current position of the robot 2. This is the end of the description of FIG.

In the sixth implementation example, the functions of the module shown in FIG. 8 are realized by the robot 2 and the data providing device 10. The voice of the user is input to the microphone 108 of the robot 2, and the data providing device 10 performs the process of voice recognition and the process of specifying execution information. For that purpose, voice data is transmitted from the robot 2 to the data providing device 10.

The robot 2 has a microphone 108 and a voice data transmission unit (not shown). The audio data transmission unit transmits the audio data input by the microphone 108 of the robot 2 to the data providing device 10. The data providing device 10 is the same as in the second implementation example.

In the seventh implementation example, the robot 2 and the data providing device 10 also implement the functions of the module shown in FIG. The robot 2 inputs a user's voice and performs voice recognition processing. The data providing device 10 performs a process of specifying execution information. For this purpose, language data is transmitted from the robot 2 to the data providing device 10.

Robot 2 has microphone 108, voice recognition unit 201, and language data transmission unit (not shown). The voice recognition unit 201 of the robot 2 converts voice data input by the microphone 108 of the robot 2 into language data. The language data transmitting unit transmits the converted language data to the data providing device 10. The data providing device 10 is the same as in the case of the fourth implementation example.

In the eighth implementation example, the functions of the modules shown in FIG. 8 are realized by the robot 2 data providing device 10. The robot 2 inputs a user's voice, and performs voice recognition processing and execution information identification processing. Then, the execution information is transmitted from the robot 2 to the data providing device 10.

The robot 2 includes the microphone 108, the voice recognition unit 201, the execution information specifying unit 202, the conversion rule storage unit 203, and the execution information transmission unit (not shown). The voice recognition unit 201 of the robot 2 converts voice data input by the microphone 108 of the robot 2 into language data. The execution information specifying unit 202 refers to the conversion rule storage unit 203 and specifies execution information based on the converted language data. The execution information transmitting unit transmits the specified execution information to the data providing device 10. The data providing device 10 is the same as in the fifth implementation example.

In the sixth to eighth implementation examples, when the execution information acquired by the voice input does not include the location information, the execution information acquisition unit 163 determines the location of the microphone 108 to which the voice is input, that is, the robot 2 A certain place may be set as the place information in the execution information. As described above, the sixth to eighth implementation examples are common in that the processing is started with the voice data acquired by the robot 2 as a starting point.

FIG. 11 is a diagram illustrating an example of a module configuration of the data providing apparatus 10 for specifying the location of the robot 2. In other words, FIG. 11 shows an example of a module configuration relating to the robot 2 and the data providing device 10 corresponding to the sixth to eighth implementation examples, and particularly to the location specifying function of the robot 2 (movable microphone 108). The robot 2 has a movement control unit 23 and a robot position transmission unit 231. The robot position transmitting unit 231 acquires the current position of the robot 2 from the movement control unit 23, and transmits the current position of the robot 2 to the data providing device 10. The movement control unit 23 may measure the current position based on a radio wave received from a wireless communication device installed at a predetermined position, or may measure the current position based on a captured image. The movement control unit 23 measures the current position of the robot 2 based on, for example, beacon signals transmitted from a plurality of beacon transmitters installed at predetermined positions indoors. In the case of this method, the robot 2 includes a beacon receiver, receives a beacon signal transmitted by a beacon transmitter installed at a predetermined position, and specifies the ID of the beacon transmitter. The movement control unit 23 specifies the current position of the robot 2 by analyzing the relationship between the radio wave intensity of the received beacon signal and the interval between the beacon transmitter specified by the ID and the robot 2. Alternatively, the movement control unit 23 may specify the current position by using a SLAM (Simultaneous Localization and Mapping) technique that simultaneously estimates its own position and creates an environment map.

In the case of the sixth implementation example, the robot position transmitting unit 231 transmits, for example, the current position of the robot 2 before or after transmitting the voice data in the voice data transmitting unit. In the case of the seventh implementation example, the robot position transmitting unit 231 transmits the current position of the robot 2 before or after the transmission of the language data in the language data transmitting unit, for example. In the case of the eighth implementation example, the robot position transmission unit 231 transmits the current position of the robot 2 before or after transmission of the execution information in the execution information transmission unit, for example.

The data providing apparatus 10 includes a robot position receiving unit 232 in addition to the place specifying unit 224 and the floor plan data storage unit 225 described above. The robot position receiving unit 232 receives the current position of the robot 2 from the robot 2. After the execution information is specified by the execution information specifying unit 202, the location specifying unit 224 determines whether or not the execution information includes the location information. When the location information is included in the execution information, the location specifying unit 224 ends the processing. When the location information is not included in the execution information, the location specifying unit 224 obtains the current position of the robot 2 from the terminal position receiving unit 223. Then, the location specifying unit 224 refers to the floor plan data storage unit 225 and writes a location including the current position of the robot 2 as location information in the execution information stored in the execution information storage unit 204. Since the location information in the execution information indicates the search location, it means that the search is performed at a location including the current position of the robot 2.

The notifying operation execution unit 26 determines the time when the execution information is obtained when the execution information obtained in each of the first to eighth implementation examples does not include time information related to the time when the notification operation is executed. The notification operation may be executed immediately. Alternatively, when the execution information acquired by the execution information acquisition unit 163 does not include the time information related to the time when the notification operation is executed, the notification operation execution unit 26 performs the predetermined time (for example, 1 to 1) from the time when the execution information is acquired. The notification operation may be executed at the point in time when the minutes have elapsed. The predetermined time is such a length as to be assumed that the character of the robot 2 understands and reacts to the instruction from the user and that the character takes the understanding and the reaction. This concludes the description of acquiring execution information by voice input.

<Provision of information on the notification target after notification>
Next, a description will be given of the provision of information on the notification target person after the notification. After executing the notification operation, the robot 2 may capture the image of the notification target person and provide the captured image from the data providing device 10 to the user terminal 3. Furthermore, the robot 2 may acquire the status information of the notification target person after executing the notification operation, and provide the status information from the data providing device 10 to the user terminal 3.

For example, when the robot 2 performs a wake-up operation with the husband user as a notification target person, the robot 2 captures the husband user with the camera of the robot 2 following the wake-up operation. The robot 2 performs the actual shooting after confirming that the user is included in the preview image obtained from the camera. Therefore, the photographed image recorded in the camera of the robot 2 by the actual photographing shows the figure of the user who is the husband after the wake-up operation. The captured image may be a still image or a moving image. The captured image is sent to the data providing device 10 and stored. On the other hand, when the user who is a wife instructs the user terminal 3 to browse the captured image in the application, the application of the user terminal 3 sends a request for the captured image after the wake-up operation to the data providing apparatus 10. The data providing device 10 sends the stored captured image to the application of the user terminal 3 in response to the request. The application of the user terminal 3 displays the received captured image on the display device of the user terminal 3. In this way, the wife user can check whether or not the husband user has woken up after the robot 2 performs the wake-up operation, by using the captured image displayed on the display device of the user terminal 3. .

FIG. 12 is a diagram illustrating an example of a module configuration of the autonomous behavior robot 1 that provides the captured image and the state information after performing the notification operation. FIG. 12 shows an example of a module configuration related to the robot 2 and the data providing apparatus 10, particularly, an information providing function. First, the provision of a captured image will be described. The robot 2 has a photographing unit 21 and a photographed image transmitting unit 241. After the robot 2 performs the notification operation, the imaging unit 21 performs imaging with the notification target person as a subject, and generates a captured image after performing the notification operation. The imaging unit 21 performs the actual imaging after confirming that the face of the notification target person is captured in the live view image by, for example, face recognition processing. The photographed image transmitting unit 241 transmits the photographed image after performing the notification operation to the data providing apparatus 10.

The data providing apparatus 10 includes a captured image receiving unit 242, a captured image storage unit 243, and a captured image providing unit 244. After the robot 2 performs the notification operation, the captured image receiving unit 242 receives the captured image sent from the robot 2, and the captured image storage unit 243 stores the received captured image. The photographed image providing unit 244 reads out the photographed image stored in the photographed image storage unit 243 and requests the photographed image after executing the notification operation from the application of the user terminal 3. To the application. In this way, in the application of the user terminal 3, it is possible to browse the captured image in which the notification target person after executing the notification operation is captured. Further, in the data providing apparatus 10, the captured image after the notification operation is performed can be stored as a record of the reaction of the notification target person.

Next, the provision of the status information will be described. For example, when the robot 2 performs a wake-up operation with the husband user as a notification target, the robot 2 acquires state information indicating that the husband user is sleeping or waking up following the wake-up operation. . The status information is sent to the data providing device 10 and stored. On the other hand, when the user who is his wife instructs the user terminal 3 to browse the state information in the application, the application of the user terminal 3 sends a request for the state information after the wake-up operation to the data providing apparatus 10. The data providing device 10 sends the stored state information to the application of the user terminal 3 in response to the request. The application of the user terminal 3 causes the display device of the user terminal 3 to display a message indicating whether the husband user is sleeping or awake according to the received state information. If the status information is sleeping, for example, a message “Dad is still sleeping.” Is displayed. If the status information is waking up, for example, a message “Dad is already up” is displayed. In this way, the wife user can confirm whether or not the husband user has woken up after the robot 2 has performed the wake-up operation, by a message displayed on the display device of the user terminal 3.

The robot 2 has a state information acquisition unit 24 and a state information transmission unit 251. After the robot 2 performs the notification operation, the state information acquisition unit 24 acquires state information relating to the state of the notification target person. As described above, the status information indicates, for example, during sleep or waking up. The state information transmitting unit 251 transmits the state information after executing the notification operation to the data providing apparatus 10.

The data providing device 10 includes a status information receiving unit 252, a status information storage unit 253, and a status information providing unit 254. After the robot 2 performs the notification operation, the state information receiving unit 252 receives the state information sent from the robot 2, and the state information storage unit 253 stores the received state information. The status information providing unit 254 reads the status information stored in the status information storage unit 253 when the status information of the notification target person after executing the notification operation is requested from the application of the user terminal 3, It is transmitted to the application of the user terminal 3. In this way, in the application of the user terminal 3, the state information on the state of the notification target person after executing the notification operation can be browsed. For example, the user of the user terminal 3 who is a wife can check whether or not the notification target person who is a husband has woken up after the robot 2 has performed the wake-up operation by the state information displayed on the display device of the user terminal 3. . Further, in the data providing apparatus 10, the state information after executing the notification operation can be stored as a record of the reaction of the notification target person.

The captured image providing unit 244 and the state information providing unit 254 may transmit the captured image and the state information to the user terminal 3 together. For example, when the user who is a wife instructs the user terminal 3 to browse the captured image and the state information in the application of the user terminal 3, the application of the user terminal 3 requests the data providing apparatus 10 for the captured image and the state information after the wake-up operation. Is sent. In response to the request, the data providing device 10 sends the stored captured image and status information to the application of the user terminal 3. The application of the user terminal 3 causes the display device of the user terminal 3 to display a message indicating whether the husband user is sleeping or waking up based on the received state information, together with the captured image. When the user who is a husband is sleeping, a photographed image of the sleeping husband and a message "Dad is still sleeping" are displayed at the same time. When the user who is a husband is awake, a photographed image showing the figure of the husband who is awake and the message "Dad is already awake" are displayed at the same time.

A live mode may be provided in which the current captured image and status information are automatically sent from the data providing device 10 to the application of the user terminal 3 so that the user terminal 3 can immediately view the image. The live mode is set by a user operation in an application of the user terminal 3. When the live mode is set in the application of the user terminal 3, a live mode setting instruction is sent to the data providing apparatus 10. When the data providing apparatus 10 receives the live mode setting instruction, it immediately transmits the captured image. That is, the captured image providing unit 244 immediately transmits the captured image received from the robot 2 and stored in the captured image storage unit 243 to the application of the user terminal 3. The application of the user terminal 3 causes the received captured image to be immediately displayed on the display device of the user terminal 3. In the case of the live mode, a message representing the current status information of the notification target person may be displayed in the application of the user terminal 3. Therefore, the state information providing unit 254 may transmit the state information received from the robot 2 and stored in the state information storage unit 253 to the application of the user terminal 3 immediately. The application of the user terminal 3 may cause the display device of the user terminal 3 to immediately display the received message indicating the status information.

In the live mode related to the wake-up operation, the captured image providing unit 244 may stop transmitting the captured image when the state information of the notification target switches from sleeping to waking up. At the same time, the status information providing unit 254 may stop transmitting the status information. In this way, unnecessary data transmission processing is omitted.

(4) When imaging the notification target person after performing the notification operation, the imaging unit 21 may change the conditions for imaging the notification target person according to the intimacy between the notification target person and the robot. A positive correlation may be provided between the intimacy between the notification target person and the robot and the shooting time. That is, the shooting time of the moving image may be set such that the higher the intimacy between the notification target person and the robot, the longer the shooting time of the moving image, and similarly, the lower the intimacy, the shorter the shooting time of the moving image. Also, a negative correlation may be provided between the intimacy between the notification target person and the robot and the shooting distance. That is, the shooting distance may be set such that the higher the intimacy between the notification target person and the robot, the shorter the shooting distance, and the lower the intimacy, the longer the shooting distance. Note that the movement control unit 23 obtains a position where the distance between the robot 2 and the notification target person is the shooting distance, and controls the movement mechanism 29 so that the robot 2 moves to that position.

<Supplementary information on the intimacy between the notification target and the robot>
In the above-described section “Intimacy between notification target person and robot”, it has been described that when the intimacy satisfies a predetermined intimacy condition, the notification operation is performed in cooperation with another robot 2. This is supplemented below.

The predetermined familiarity condition may be a condition that the familiarity is equal to or higher than a reference value. The predetermined intimacy condition may be a condition that the intimacy level is less than a reference. Further, the condition may include a relationship between the intimacy between the other robot 2 and the notification target person. For example, the familiarity condition may be that the familiarity A between the robot 2a and the subject to be notified is a value obtained by multiplying the familiarity B between the other robot 2b and the subject to be notified by a predetermined ratio. That is, a result of comparing the intimacy with which some or all of the plurality of robots 2 are weighted may be used as the intimacy condition.

<Notification operation in which two robots 2 cooperate>
Three specific examples of the notification operation in which two robots 2 cooperate are shown. In each specific example, the robot 2a and the robot 2b operate in the same room.

FIG. 13 is a diagram showing a first specific example of a notification operation in which two robots 2 cooperate. In the first specific example, the robot 2a that has found the notification target notifies the robot 2b of the discovery location, and the notified robot 2b moves to the discovery location and performs a notification operation. That is, when the robot 2a performs the notification operation, the robot 2b is called in, and the called robot 2b also performs an effect to perform the notification operation together with the robot 2a.

For example, when the robot 2a discovers the notification target person sleeping in the bedroom and tries to wake up the notification target person, the robot 2b detects that the notification target person is found in the bedroom. Notice. Thereafter, the robot 2a starts a wake-up operation for the notification target person. On the other hand, when the robot 2b receives the notification that the notification target person has been found in the bedroom, the robot 2b starts moving to the bedroom. When arriving at the bedroom, the robot 2b starts an alarming operation for the notification target together with the robot 2a. Therefore, the number of the robots 2 performing the wake-up operation increases from one to two, and the effect of wake-up on the notification target person increases from the middle. Therefore, even if the notification target person falls asleep deeply and does not easily wake up, the notification target person can be easily woken up.

The robot 2a has a discovery location notification transmission unit (not shown) for transmitting a notification of the location where the notification target is found, that is, a notification of the discovery location, to the robot 2b, and the robot 2b transmits a notification of the discovery location from the robot 2a. It has a discovery location notification receiving unit (not shown) for receiving.

In step S31, as described in step S21 of FIG. 5 and the section of "acquisition of execution information by voice input", when execution information acquisition section 163 of robot 2a acquires execution information, in step S32, step S22 of FIG. As described in, the search unit 25 of the robot 2a calculates the movement route to the search location. Then, in step S33, as described in step S23 of FIG. 5, the search unit 25 of the robot 2a starts searching for a notification target person, and the movement control unit 23 of the robot 2a controls the movement mechanism 29 to search for the search location. Start moving to.

In step S34, as described in step S24 of FIG. 5, the movement control unit 23 of the robot 2a determines that the robot 2a has reached the search location, and further in step S35, as described in step S25 of FIG. When the search unit 25 of the robot 2a finds the notification target person, the discovery location notification transmission unit of the robot 2a transmits a notification of the discovery location to the robot 2b (step S36). Then, in step S37, the notification operation execution unit 26 of the robot 2a performs the notification operation as exemplified in steps S27 to S29 of FIG.

When the discovery location notification receiving unit of the robot 2b receives the notification of the discovery location from the robot 2a, the search unit 25 of the robot 2b calculates a moving route to the discovery location (step S41). Then, the movement control unit 23 of the robot 2b controls the movement mechanism 29 to start moving to the discovery location (Step S42). When the movement control unit 23 of the robot 2b determines that the robot 2b has reached the discovery location (step S43), the notification operation execution unit 26 of the robot 2b performs a notification operation (step S44).

In the example of FIG. 13, only the robot 2a performs the search and the robot 2b does not perform the search. However, the robot 2b may perform the search simultaneously with the robot 2a. Then, when the robot 2b first finds the notification target person, the robot 2b transmits a discovery location notification to the robot 2a, and the robot 2a that has received the discovery location notification moves to the discovery location, contrary to the example of FIG. Then, the notification operation may be performed. For this purpose, the robot 2b may further include a discovery location notification transmission unit, and the robot 2a may further include a discovery location notification reception unit.

The discovery location notification may be transmitted via the data providing device 10. For example, the discovery location notification transmitting unit of the robot 2a transmits a discovery location notification to the data providing device 10, and the discovery location notification transferring unit (not shown) of the data providing device 10 receives and receives the discovery location approach notification. The discovery location notification may be transferred to the robot 2b. The discovery location notification receiving unit of the robot 2b may receive the transferred discovery location notification.

FIG. 14 is a diagram showing a second specific example of the notification operation in which two robots 2 cooperate. In the second specific example, when both of the robots 2a and 2b approach the notification target person to a distance equal to or less than a predetermined reference, the robots 2a and 2b perform the notification operation in synchronization. Hereinafter, approaching the notification target person to a distance equal to or smaller than a predetermined reference is expressed as “approaching the notification target person”.

In the example of FIG. 14, the robot 2a that has approached the notification target first waits until the robot 2b approaches the notification target. When the robot 2b approaches the notification target person, the robots 2a and 2b perform the notification operation at the same time.

For example, if the robot 2a finds a notification target person sleeping in the bedroom, it approaches the notification target person. Then, the robot 2a notifies the robot 2b that it has approached the notification target person. The robot 2a waits without performing an alarming operation until the robot 2b approaches the notification target person. On the other hand, the robot 2b also finds the notification target person sleeping in the bedroom and approaches the notification target person. Then, when both the robots 2a and 2b approach the notification target, the robots 2a and 2b simultaneously start an alarming operation on the notification target. Therefore, the alarming effect is strong, and the notification target person can be easily caused to wake up. In addition, it is possible to produce a situation in which the robots 2a and 2b are aligned and play mischief.

The robot 2a receives an approach notification from the other robot 2b (not shown) and an approach notification transmitting unit (not shown) that sends a notification (hereinafter, referred to as “approach notification”) of the approach to the notification target person to the other robot 2b. And an approach notification receiving unit (not shown). Similarly, the robot 2b also has an approach notification transmitting unit and an approach notification receiving unit.

処理 The processing of the robot 2a shown in steps S51 to S54 of FIG. 14 is the same as the processing of the robot 2a shown in steps S31 to S34 of FIG. When the search unit 25 of the robot 2a finds the notification target person and determines that the notification target person is approached (step S55), the approach notification transmitting unit transmits an approach notification to the robot 2b (step S56). Then, the robot 2a waits until receiving the approach notification from the robot 2b.

処理 The processing of the robot 2b shown in steps S61 to S63 of FIG. 14 is the same as the processing of the robot 2a shown in steps S31 to S33 of FIG. It is assumed that the robot 2b receives an approach notification from the robot 2a before arriving at the search location. Therefore, after the approach notification receiving unit of the robot 2b receives the approach notification from the robot 2a, the movement control unit 23 of the robot 2b determines that the robot 2a has reached the search location (step S64). Further, when the search unit 25 of the robot 2b finds the notification target person and determines that the notification target person is approached (step S65), the approach notification transmission unit of the robot 2b transmits an approach notification to the robot 2a (step S66). . The notification operation execution unit 26 of the robot 2b determines that the predetermined cooperation condition is satisfied, and performs the notification operation. The predetermined cooperation condition is that the approach notification is received from the other robot 2a and that the own robot 2b approaches the notification target.

When the approach notification receiving unit of the robot 2a receives the approach notification from the robot 2b, the notification operation execution unit 26 of the robot 2a determines that the predetermined cooperation condition is satisfied, and performs the notification operation. The predetermined cooperation condition is that an approach notification is received from the other robot 2b, and that the own robot 2a approaches the notification target person.

The approach notification may be transmitted via the data providing device 10. For example, the approach notification transmitting unit of the robot 2a transmits an approach notification to the data providing device 10, and the approach notification transferring unit (not shown) of the data providing device 10 receives the approach notification and transmits the received approach notification to the robot 2b. May be forwarded to The approach notification receiving unit of the robot 2b may receive the transferred approach notification.

FIG. 15 is a diagram showing a third specific example of the notification operation in which two robots 2 cooperate. In the third specific example, the timing of the notification operation by the robot 2a is shifted from the timing of the notification operation by the robot 2b. Therefore, the timings of the notification operations by the two robots 2a and 2b do not overlap. For example, since the voice output of the robot 2a and the voice output of the robot 2b are not performed at the same time, it is not noisy and the content of the voice can be easily heard.

The third specific example relates to the control of the stage where both the robots 2a and 2b perform the notification operation. That is, in the first specific example, the notification operation shown in step S44 of FIG. 13 is started, or in the second specific example, the notification operation shown in step S57 of FIG. 14 and the notification operation shown in step S67 are started. It relates to control after the timing.

The robot 2a receives an operation notification from the other robot 2b, and an operation notification transmitting unit (not shown) that transmits a notification (hereinafter, referred to as “operation notification”) that the robot 2a has performed the notification operation to the other robot 2b. And an operation notification receiving unit (not shown). Similarly, the robot 2b has an operation notification transmitting unit and an operation notification receiving unit.

In the example of FIG. 15, the robot 2a first performs the notification operation, and then performs the notification operation twice each while replacing the robot 2b. First, the notification operation execution unit 26 of the robot 2a performs a notification operation (step S71). When the notification operation is completed, the operation notification transmission unit of the robot 2a transmits an operation notification to the robot 2b (Step S72). Then, the robot 2a waits until an operation notification is received from the robot 2b.

When the operation notification receiving section of the robot 2b receives the operation notification from the robot 2a (step S81), the robot 2b waits for a predetermined time and the notification operation executing section 26 of the robot 2b performs the notification operation (step S82). The predetermined waiting time is an interval (for example, 1 second to 5 seconds) that gives an impression that the robots 2a and 2b repeat the notification while observing the state of the notification target person without folding. When the notification operation is completed, the operation notification transmitting unit of the robot 2b transmits an operation notification to the robot 2a (Step S83). Then, the robot 2b waits until receiving the operation notification from the robot 2a.

(4) Thereafter, when the operation notification receiving unit of the robot 2a receives the operation notification from the robot 2b (step S73), the operation waits for a predetermined time, and the notification operation execution unit 26 of the robot 2a performs the notification operation (step S74). When the notification operation is completed, the operation notification transmitting unit of the robot 2a transmits an operation notification to the robot 2b (Step S75). Then, the robot 2a ends the entire process.

(4) The robot 2b repeats the same processing as the processing in steps S81 to S83 (steps S84 to S86), and ends the entire processing.

For example, when the notification operation is a wake-up operation, both the robots 2a and 2b may end the process of the third specific example at the timing when it is determined that the state information of the notification target person is awake.

For example, if the first wake-up operation by the robot 2a is performed after the first wake-up operation by the robot 2a, and the second wake-up operation by the robot 2a is performed, and the notification target person wakes up, The second wake-up operation by 2b is not performed. That is, an unnecessary alarm operation is not performed on the notification target person who has already woken up.

<Notification operation by message reception>
When the autonomous behavior robot 1 receives a message addressed to a user, a notification operation may be performed with the user as a notification target. Hereinafter, four examples of the system of the notification operation by the message reception will be described.

First, a specific example of the first method will be described. The parent user terminal 3 sends an e-mail with the message "You can eat cake in the refrigerator" in the text to the user who is a child. The robot 2 can read out a message to a child who is in the robot. In this example, it is assumed that the child's e-mail address and the child's search location “child room” are set in advance as the child user information.

The e-mail address of the child is set as the destination by the parent, and when the user terminal 3 transmits an e-mail in which the message "Eat cake in the refrigerator." Is sent, the autonomous robot 1 This e-mail is received at.

(4) The autonomous behavior robot 1 determines that the notification target is a child from the destination e-mail address, and the robot 2 moves to the “child room” where the child is to be searched and searches for the child. When the robot 2 finds a child, the robot 2 reads out the text of the e-mail address, "You can eat the cake in the refrigerator."

As described above, in the first method, the notification target person is specified based on the email address, and the search location of the notification target person is specified based on the user information of the notification target person. Then, the main body of the e-mail is read out as a notification operation.

Next, a specific example of the second method will be described. By transmitting an e-mail with a place expression of “＠bed” in the text from the wife's user terminal 3 to the user who is a husband, a predetermined notification operation “wake-up operation” to the husband in the bedroom is performed. This can be performed by the robot 2. In this example, it is assumed that there is a message description rule that a place expression designating a search place is described after "@". In this example, it is assumed that the husband's e-mail address and the notification operation “wake-up operation” for the husband are set in advance as the husband's user information.

When the wife's user terminal 3 sends an e-mail with the husband's e-mail address set as the destination by the wife and a place expression of "@bed" in the text, the e-mail is sent to the autonomous robot 1. Received.

The autonomous robot 1 determines from the e-mail address of the destination that the person to be notified is her husband, and specifies the search location “bedroom” corresponding to the location expression “＠bed” from the text of the e-mail address. Further, the autonomous behavior robot 1 specifies a notification operation “alarm operation” for the husband based on the user information of the husband. Then, the robot 2 moves to the “bedroom” that is a search place and searches for a husband. When the robot 2 finds the husband, the robot 2 performs a notification operation “alarm operation” to the husband.

As described above, in the second method, the notification target person is specified based on the e-mail address, the search location is specified from the body of the e-mail, and the notification operation for the notification target person is specified based on the user information of the notification target person. Is done.

Next, a specific example of the third method will be described. From the wife's user terminal 3, an e-mail with the message “Please call me. Bed” is sent to the user who is my husband, so that the message “Please call me” to my husband in the bedroom is sent. The reading can be performed by the robot 2. As in the second specific example, it is assumed that there is a message description rule in which a place expression designating a search place is described after "@". The robot 2 reads out the text before "@" and does not read out the place expression. In this example, it is assumed that the husband's e-mail address is set in advance as the husband's user information.

When the wife's user terminal 3 sends an e-mail with the wife's e-mail address set as the destination and the message "Please call me. Bed" in the body, the autonomous robot 1 sends this e-mail. Email is received.

The autonomous behavior robot 1 determines that the notification target person is the husband from the destination e-mail address, and specifies the search location “bedroom” corresponding to the location expression “＠bed” in the body of the e-mail address. Then, the robot 2 moves to the “bedroom” that is a search place and searches for a husband. When the robot 2 finds the husband, it reads out the text "please call" except for the location expression "$ bed" in the body of the e-mail address.

Thus, in the third method, the notification target person is specified based on the e-mail address, and the search location is specified from the body of the e-mail. Then, the main body of the e-mail is read out as a notification operation.

Finally, a specific example of the fourth method will be described. By transmitting an e-mail from the wife's user terminal 3 to the user who is the husband, the robot 2 can perform a predetermined notification operation “alarm operation” for the husband in the predetermined search location “bedroom”. it can. In this example, since the robot 2 does not refer to the text of the e-mail when performing the notification operation, the text of the e-mail is arbitrary. The body of the email may be empty. In this example, it is assumed that an e-mail address of the husband, a search location "bedroom" of the husband, and a notification operation "wake-up operation" for the husband are set in advance as the user information of the husband.

(4) When the wife's user terminal 3 transmits an e-mail with the husband's e-mail address set as the destination by the wife, the autonomous robot 1 receives the e-mail.

(4) The autonomous behavior robot 1 determines that the person to be notified is her husband from the destination e-mail address. Further, the autonomous behavior robot 1 specifies a search location “bedroom” of the husband and a notification operation “alarm operation” for the husband based on the user information of the husband. Then, the robot 2 moves to the “bedroom” that is a search place and searches for a husband. When the robot 2 finds the husband, the robot 2 performs a notification operation “alarm operation” to the husband.

As described above, in the fourth method, the notification target person is specified based on the electronic mail address, and the search location of the notification target person and the notification operation for the notification target person are specified based on the user information of the notification target person.

Note that the message transmission method is not limited. The message may be transmitted by a method other than e-mail. For example, the message may be transmitted by a message exchange application.

Next, details of the notification operation by receiving a message will be described. Hereinafter, the first to fourth schemes will be comprehensively described first, and the features of each scheme will be described later.

The user information is set by, for example, an application of the user terminal 3. The set user information is transmitted to the data providing device 10 by a user information transmitting unit (not shown) in the application of the user terminal 3.

The data providing device 10 includes a user information receiving unit (not shown), a user information storage unit (not shown), a message receiving unit (not shown), and a notification target specifying unit (not shown). The user information received by the user information receiving unit is stored in the user information storage unit. The user information storage unit stores user information such as user identification information for message communication, a search location, and a type of notification operation in association with the user. The user identification information for message communication is, for example, an e-mail address or a message exchange application ID. The user information storage unit stores, as user information, information about the user, such as the name of the user, information indicating the physical characteristics of the user, personal belongings, clothes, intimacy with the robot 2, and the like. You may. Information indicating the physical characteristics of the user includes, for example, information for recognizing the face of the notification target, information for recognizing the fingerprint of the notification target, information for recognizing the physique of the notification target, and the like. It is. Note that the robot 2 may include a user information receiving unit, a user information storage unit, a message receiving unit, and a notification target person specifying unit. In that case, the user information transmitting unit in the application of the user terminal 3 may transmit the user information to the robot 2.

FIG. 16 is a flowchart showing a notification operation by receiving a message. When the message receiving unit receives a message addressed to the user of the robot 2 (step S71), the notification target person specifying unit refers to the user information storage unit and sets a message communication destination set for the message. The user corresponding to the user identification information is specified as a notification target person (step S72).

The search unit 25 refers to the user information storage unit and specifies a search location corresponding to the user who is the notification target (step S73). The search unit 25 may specify the search location based on the location expression included in the received message. For example, when the location expression “場所 bed” is included in the message, the search unit 25 may specify “bedroom” as the search location.

In step S74, as described in step S22 of FIG. 5, the search unit 25 calculates the movement route to the search location. Then, in step S75, as described in step S23 of FIG. 5, the search unit 25 starts searching for a notification target person, and the movement control unit 23 controls the moving mechanism 29 to start moving to the search location.

In step S76, as described in step S24 of FIG. 5, the movement control unit 23 determines that the robot 2 has reached the search location, and further in step S77, as described in step S25 of FIG. Finds the notification target person, in step S78, the notification operation execution unit 26 performs the notification operation. The notification operation in this example is, for example, reading out a received message. The type of the notification operation may be indicated by the received message. The notification operation execution unit 26 may specify the type of the notification operation corresponding to the user who is the notification target person by referring to the user information storage unit.

In the case of the first method, in step S73, the search unit 25 refers to the user information storage unit and specifies a search location corresponding to the user who is the notification target. Further, in step S78, the notification operation execution unit 26 reads out the received message.

In the case of the second method, in step S73, the search unit 25 specifies a search location based on a location expression included in the received message. Further, in step S78, the notification operation executing unit 26 specifies the type of the notification operation corresponding to the user who is the notification target person with reference to the user information storage unit.

In the case of the third method, in step S73, the search unit 25 specifies a search location based on a location expression included in the received message. Further, in step S78, the notification operation execution unit 26 reads out the received message.

In the case of the fourth method, in step S73, the search unit 25 refers to the user information storage unit and specifies a search location corresponding to the user who is the notification target. Further, in step S78, the notification operation executing unit 26 specifies the type of the notification operation corresponding to the user who is the notification target person with reference to the user information storage unit.

The first to fourth methods may be arbitrarily combined. For example, the first method and the second method may be combined. The first method and the third method may be combined. The first method and the fourth method may be combined. The second method and the third method may be combined. The second method and the fourth method may be combined. The third method and the fourth method may be combined. The first method, the second method, and the third method may be combined. The first method, the second method, and the fourth method may be combined. The first method, the third method, and the fourth method may be combined. The second system, the third system, and the fourth system may be combined. The first method, the second method, the third method, and the fourth method may be combined.

In connection with the combination of the above-described methods, in step S73, when the received message includes the location expression, the search unit 25 specifies the search location by the location expression, and the received message includes the location expression. If not, the search location corresponding to the user to be notified may be specified by referring to the user information storage unit. In step S78, when the content of the received message is empty, the notification operation execution unit 26 refers to the user information storage unit and specifies the type of the notification operation corresponding to the user who is the notification target. If the content of the received message is not empty, the received message may be read aloud.

Claims

An execution information acquisition unit that acquires execution information for performing a notification operation of information to be performed on a notification target person,
Based on the execution information acquired in the execution information acquisition unit, a search unit to search for the notification target person,
A notification operation execution unit that executes the notification operation based on the execution information for the notification target person searched by the search unit.
2. The robot according to claim 1, wherein the execution information acquisition unit acquires location information related to a location designated by a user as the execution information. 3.
The said execution information acquisition part acquires the said location information designated by the said user operating the map displayed on the user terminal operated by the said user from the said user terminal, The Claim 2 characterized by the above-mentioned. Robot.
4. The robot according to claim 1, wherein the search unit searches for the notification target person further based on a captured image of a surrounding space. 5.
The robot according to claim 4, wherein the search unit searches for the notification target person by recognizing a person included in the captured image.
A movement control unit that controls the movement mechanism,
The search unit calculates a moving route by the moving mechanism based on the execution information,
The robot according to claim 1, wherein the movement control unit controls the movement mechanism based on a movement path calculated by the search unit.
The robot according to claim 6, wherein the search unit calculates the movement path further based on restriction information for restricting movement by the moving mechanism.
The image capturing apparatus further includes a marker recognition unit that recognizes a predetermined marker included in a captured image of a surrounding space,
The robot according to claim 6, wherein the movement control unit controls the movement mechanism based on the marker recognized by the marker recognition unit.
The information processing apparatus further includes a state information acquisition unit that acquires state information relating to a state of the notification target person searched in the search unit,
The robot according to any one of claims 1 to 8, wherein the notification operation execution unit changes the notification operation according to the state acquired by the state information acquisition unit.
The state information acquisition unit acquires the state whether the notification target person is sleeping or waking up,
The notification operation execution unit, as the notification operation,
When the state is sleeping, execute a wake-up operation to wake up the notification target person,
The robot according to claim 9, wherein when the state is waking up, the robot performs a greeting operation for the notification target person.
The execution information acquisition unit acquires the execution information associated with the notification target person,
The robot according to any one of claims 1 to 10, wherein the notifying operation executing unit executes the notifying operation associated with the searched notification target person.
The execution information acquisition unit acquires intimacy information indicating the level of intimacy between the notification target person and the robot as the execution information,
The informing operation executing unit, when the intimacy in the intimacy information acquired by the execution information acquiring unit satisfies a predetermined intimacy condition, executes the informing operation in cooperation with another robot, The robot according to any one of claims 1 to 11.
The execution information acquisition unit acquires the execution information associated with a plurality of the notification target persons,
The robot according to any one of claims 1 to 12, wherein the notification operation execution unit executes the notification operations associated with the plurality of notification target persons in parallel.
The execution information obtaining unit obtains time information relating to a time for executing the notification operation as the execution information,
The robot according to any one of claims 1 to 13, wherein the notification operation execution unit executes the notification operation based on the time information.
It further includes a voice recognition unit that recognizes voice input to the microphone and converts it into language data,
The robot according to claim 1, wherein the execution information acquisition unit specifies the execution information based on the converted language data.
The notifying operation executing unit, when the acquired execution information does not include time information related to a time for executing the notifying operation, the notifying operation is performed at a point in time when a predetermined time has elapsed from the point in time when the executing information was obtained. The robot according to claim 15, wherein the robot performs:
17. The robot according to claim 15, wherein, when the execution information does not include location information, the execution information acquisition unit sets the location of the microphone to which the voice is input as location information.
After performing the notification operation, a photographing unit that photographs the notification target person,
The robot according to any one of claims 1 to 17, further comprising: a transmission unit configured to transmit image data of the notification target person to the user terminal.
19. The robot according to claim 18, wherein the photographing unit changes a condition for photographing the notification target person according to a degree of closeness between the notification target person and the robot.
After performing the notification operation, a state information acquisition unit that acquires state information related to the state of the notification target person,
The robot according to any one of claims 1 to 19, further comprising: a transmission unit configured to transmit the obtained state information to a user terminal.
A message receiving unit for receiving a message addressed to the robot user;
A notification target person specifying unit that specifies a notification target person by a destination of the received message,
At a place specified by the message or a place specified in correspondence with the notification target person, a search unit that searches for the notification target person,
A notification unit that reads out the message or performs a notification operation instructed by the message with respect to the notification target person searched by the search unit.
In the robot,
An execution information obtaining step of obtaining execution information for executing a notification operation of information to be performed on a notification target person,
A search step of searching for the notification target person based on the execution information acquired in the execution information acquisition step,
A notification operation execution step of executing the notification operation based on the execution information for the notification target person searched in the search step.
To the robot,
Execution information acquisition processing for acquiring execution information for executing a notification operation of information to be performed on a notification target person;
Based on the execution information acquired in the execution information acquisition process, a search process for searching for the notification target person,
A robot control program for causing the notification target person searched in the search process to execute a notification operation execution process of executing the notification operation based on the execution information.
In the robot,
A message receiving step of receiving a message addressed to a user of the robot;
A notification target person specifying step of specifying a notification target person by a destination of the received message,
At a place specified by the message or at a place specified corresponding to the notification target, a search step of searching for the notification target,
A notification step of reading out the message or performing a notification operation instructed by the message to the notification target person searched in the search step.
To the robot,
A message receiving process for receiving a message addressed to a user of the robot;
Notification target person specifying processing for specifying a notification target person by the destination of the received message,
At a place specified by the message or at a place specified corresponding to the notification target person, a search process of searching for the notification target person,
A robot control program for causing the notification target person searched in the search process to execute a reading process of reading out the message or a notification operation instructed by the message.