JP2022083725A - Management device, management system, management method, and program - Google Patents

Abstract

To surely guide an object person taken by a protector to a destination.SOLUTION: A management device for managing a robot device includes: an acquisition part for acquiring identification information of an object person who is a protection object made by a protector, and confirmation information for confirming that the object person is taken by the protector to reach a meeting place from a vehicle or a terminal device; and a provision part for providing the robot device with instruction information including the identification information and the confirmation information for causing the robot device to guide the object person from the meeting place to a destination on the basis of the identification information and the confirmation information acquired by the acquisition part.SELECTED DRAWING: Figure 1

Description

The present invention relates to a management device, a management system, a management method, and a program.

Conventionally, a storage unit that collects and stores route information including a barrier-free route traveled by a guide vehicle that discriminates and travels on a barrier-free route from the guide vehicle, a user's standby position, and a user. A receiving unit that receives the target position, an acquisition unit that acquires the route information from the standby position to the target position from the route information stored by the storage unit, and the route acquired by the acquisition unit. A route management device including a providing unit that provides information as a barrier-free route from the standby position to the target position is disclosed (see, for example, Patent Document 1).

Japanese Unexamined Patent Publication No. 2020-91142

However, the above system does not consider guiding the target person taken by the guardian to the destination.

The present invention has been made in consideration of such circumstances, and provides a management device, a management system, a management method, and a program capable of reliably guiding a target person taken by a guardian to a destination. One of the purposes is to do.

The management device, management system, management method, and program according to the present invention have adopted the following configurations.
(1): The management device according to one aspect of the present invention is a management device that manages a robot device, and protects the identification information of the target person to be protected by the guardian from the vehicle or the terminal device and the target person. Based on the acquisition unit that acquires confirmation information for confirming that the person has arrived at the meeting place, the identification information acquired by the acquisition unit, and the confirmation information, the target person is arranged for the meeting. It is a management device including a providing unit that provides the robot device with instruction information including the identification information and the confirmation information for guiding the robot device from a place to a destination.

(2): In the embodiment of (1) above, the confirmation information includes an image captured by the guardian, feature information indicating features extracted from the image captured by the guardian, and a voice of the guardian. The feature information indicating the feature, the position information of the vehicle, the position information of the terminal device, and the preset password.

(3): In the aspect of (1) or (2) above, the identification information of the subject is the feature information indicating the characteristics extracted from the image captured by the subject and the image captured by the subject. , Feature information indicating the characteristics of the voice of the subject, position information of the device held by the subject, or a preset password.

(4): In any of the above embodiments (1) to (3), the instruction information causes the robot device to stand by at a time when it is scheduled to arrive near the meeting place, and the subject makes the meeting. Includes instructions to guide the subject to the destination after arriving at the location.

(5): In any of the above embodiments (1) to (4), the providing unit provides completion information indicating that the robot device has guided the target person to the destination. When obtained from the device, the completion information is provided to the vehicle or terminal device.

(6): In any of the above embodiments (1) to (5), the providing unit transfers information indicating the characteristics of the robot device that guides the target person to the destination to the vehicle or the terminal device. offer.

(7): Based on the management device according to any one of the above (1) to (6) and the instruction information provided by the providing unit of the management device, the subject is moved from the meeting place to the destination. It is a management system equipped with a robot device that guides.

(8): In the embodiment of the above (7), the robot device acquires the instruction information and the confirmation information including the identification information from the management device, the process based on the confirmation information is completed, and the meeting place. When the identification information input by the target person arriving at is matched with the identification information provided by the providing unit, the target person is guided from the meeting place to the destination.

(9): In the embodiment (7) or (8), the robot device acquires the estimated time of arrival of the target person from the vehicle or the terminal device, and waits at the meeting place at the estimated time of arrival. ..

(10): In the management system according to any one of the above (7) to (9), when the robot device guides the target person from the meeting place to the destination, the robot device is connected to the robot device. Information that indicates that the abnormality has occurred by analyzing the image captured by the on-board imaging device to determine whether or not the abnormality has occurred in the subject, and if it is determined that the abnormality has occurred. To the vehicle or terminal device.

(11): In the management system according to any one of the above (7) to (10), when the robot device guides the target person from the meeting place to the destination, the robot device is connected to the robot device. Information that indicates that the abnormality has occurred by analyzing the image captured by the on-board imaging device to determine whether or not the abnormality has occurred in the subject, and if it is determined that the abnormality has occurred. Is output to the output unit mounted on the robot device.

(12): In the management system according to any one of the above (7) to (11), the identification information of the target person to be protected by the guardian and the target person are taken to the meeting place by the guardian. Further includes a vehicle for transmitting confirmation information for confirming arrival to the management device.

(13): The management device according to any one of the above (1) to (6), the identification information of the target person to be protected by the guardian, and the target person being taken to the meeting place by the guardian. It is a management system including a vehicle for transmitting confirmation information for confirming arrival to the management device.

(14): The management method according to one aspect of the present invention is a management method for managing a robot device, in which a computer identifies information of a target person to be protected by a guardian from a vehicle or a terminal device, and the target. The confirmation information indicating that the person has arrived at the meeting place with the guardian is acquired, and the target person is moved from the meeting place to the destination based on the acquired identification information and the confirmation information. It is a management method that provides the robot device with instruction information including the identification information and the confirmation information for guiding the robot device.

(15): The program according to one aspect of the present invention is a program for managing a robot device, and includes identification information of a target person protected by a guardian from a vehicle or a terminal device on a computer and the target. The person is made to acquire confirmation information indicating that the person has arrived at the meeting place with the guardian, and based on the acquired identification information and the confirmation information, the target person is moved from the meeting place to the destination. This is a program for causing the robot device to provide instruction information including the identification information and the confirmation information for guiding the robot device.

According to (1) to (15), the management device waits for the target person based on the identification information of the target person and the confirmation information indicating that the target person has arrived at the meeting place with the guardian. By providing the robot device with instruction information including identification information and confirmation information for guiding the robot device from the place to the destination, it is possible to reliably guide the target person taken by the guardian to the destination. can.

According to (5), the management device improves the sense of security of the guardian by providing the vehicle or the terminal device with the completion information indicating that the robot device has guided the target person to the destination. be able to.

According to (6), the management device provides information indicating the characteristics of the robot device that guides the target person to the destination to the vehicle or the terminal device, so that the guardian or the target person makes the target person the destination. The guiding robot device can be easily recognized.

According to (10), when the robot device guides the target person to the meeting place and an abnormality occurs, the robot device provides information indicating that the abnormality has occurred to the vehicle or the terminal device. Parents can quickly recognize the anomaly.

According to (11), when the robot device guides the target person to the meeting place, information indicating that an abnormality has occurred is output to the output unit mounted on the robot device, thereby promptly surrounding the surroundings. It is possible to notify the occurrence of an abnormality.

It is a figure which shows an example of the structure of the management system 1 including the management apparatus. It is a block diagram of the vehicle system 2. It is a figure which shows an example of the functional structure of management apparatus 500. It is a figure which shows an example of the functional structure of a robot apparatus 600. It is a figure for demonstrating the outline of the service provided by management system 1. It is a figure which shows an example of the flow of processing executed by the processing unit 170 of a vehicle M. It is a figure which shows an example of the image IM1 displayed on the HMI30 of a vehicle M. It is a flowchart which shows an example of the flow of the process which the management apparatus 500 executes. It is a figure which shows an example of the information (information before processing, service information 522) which the management apparatus 500 acquired from a vehicle M, and the information (information after processing, task management information 632) transmitted to the robot apparatus 600. It is a flowchart which shows an example of the flow of the process executed in the robot apparatus 600. It is a flowchart which shows an example of the flow of processing executed by a robot device, a management device, and a vehicle. It is a flowchart which shows an example of the flow of the process executed in the robot apparatus 600 of 2nd Embodiment.

Hereinafter, embodiments of the management device, management system, management method, and program of the present invention will be described with reference to the drawings.

<First Embodiment>
[overall structure]
FIG. 1 is a diagram showing an example of a configuration of a management system 1 including a management device. The management system 1 includes, for example, a vehicle M, a terminal device 400, a management device 500, and a robot device 600. These communicate via the network NW. The network NW includes the Internet, WAN (Wide Area Network), LAN (Local Area Network), public line, provider device, dedicated line, wireless base station and the like.

[vehicle]
FIG. 2 is a configuration diagram of the vehicle system 2. The vehicle on which the vehicle system 2 is mounted is, for example, a vehicle such as a two-wheeled vehicle, a three-wheeled vehicle, or a four-wheeled vehicle, and the drive source thereof is an internal combustion engine such as a diesel engine or a gasoline engine, an electric motor, or a combination thereof. The electric motor operates by using the electric power generated by the generator connected to the internal combustion engine or the electric power generated by the secondary battery or the fuel cell.

The vehicle system 2 includes, for example, a camera 10, a radar device 12, a finder 14, an object recognition device 16, a communication device 20, an HMI (Human Machine Interface) 30, a vehicle sensor 40, a navigation device 50, and the like. MPU (Map Positioning Unit) 60, driving controller 80, automatic driving control device 100, traveling driving force output device 200, braking device 210, steering device 220, agent device 300, vehicle interior camera 310. To prepare for. These devices and devices are connected to each other by a multiplex communication line such as a CAN (Controller Area Network) communication line, a serial communication line, a wireless communication network, or the like. The configuration shown in FIG. 1 is merely an example, and a part of the configuration may be omitted or another configuration may be added.

The camera 10 is, for example, a digital camera using a solid-state image sensor such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor). The camera 10 is attached to an arbitrary position on the vehicle (hereinafter, vehicle M) on which the vehicle system 2 is mounted. The radar device 12 radiates radio waves such as millimeter waves around the vehicle M, and also detects radio waves (reflected waves) reflected by the object to detect at least the position (distance and direction) of the object. The finder 14 is a LIDAR (Light Detection and Ranging). The finder 14 irradiates the periphery of the vehicle M with light and measures the scattered light. The finder 14 detects the distance to the target based on the time from light emission to light reception.

The object recognition device 16 performs sensor fusion processing on the detection results of a part or all of the camera 10, the radar device 12, and the finder 14, and recognizes the position, type, speed, and the like of the object. The object recognition device 16 outputs the recognition result to the automatic operation control device 100.

The communication device 20 communicates with other vehicles existing in the vicinity of the vehicle M by using, for example, a network NW, Bluetooth (registered trademark), DSRC (Dedicated Short Range Communication), or various types via a wireless base station. Communicate with the server device.

The HMI 30 presents various information to the occupant of the vehicle M and accepts input operations by the occupant. The HMI 30 includes various display devices, speakers, buzzers, touch panels, switches, keys and the like.

The vehicle sensor 40 includes a vehicle speed sensor that detects the speed of the vehicle M, an acceleration sensor that detects the acceleration, a yaw rate sensor that detects the angular velocity around the vertical axis, an orientation sensor that detects the direction of the vehicle M, and the like.

The navigation device 50 includes, for example, a GNSS (Global Navigation Satellite System) receiver 51, a navigation HMI 52, and a routing unit 53. The navigation device 50 holds the first map information 54 in a storage device such as an HDD (Hard Disk Drive) or a flash memory. The GNSS receiver 51 identifies the position of the vehicle M based on the signal received from the GNSS satellite. The navigation HMI 52 includes a display device, a speaker, a touch panel, keys, and the like. The navigation HMI 52 may be partially or wholly shared with the above-mentioned HMI 30. The route determination unit 53, for example, is a route from the position of the vehicle M specified by the GNSS receiver 51 (or an arbitrary position input) to the destination input by the occupant using the navigation HMI 52 (hereinafter, a map). The upper route) is determined with reference to the first map information 54. The first map information 54 is, for example, information in which a road shape is expressed by a link indicating a road and a node connected by the link. The first map information 54 may include road curvature, POI (Point Of Interest) information, and the like. The navigation device 50 may be realized by, for example, the function of a terminal device such as a smartphone or a tablet terminal owned by an occupant. The navigation device 50 may transmit the current position and the destination to the navigation server via the communication device 20 and acquire a route equivalent to the route on the map from the navigation server.

The MPU 60 includes, for example, a recommended lane determination unit 61, and holds the second map information 62 in a storage device such as an HDD or a flash memory. The recommended lane determination unit 61 divides the route on the map provided by the navigation device 50 into a plurality of blocks (for example, divides the route into 100 [m] units with respect to the vehicle traveling direction), and refers to the second map information 62. Determine the recommended lane for each block. The recommended lane determination unit 61 determines which lane to drive from the left.

The second map information 62 is map information with higher accuracy than the first map information 54. The second map information 62 includes, for example, information on the center of the lane, information on the boundary of the lane, and the like. Further, the second map information 62 may include road information, traffic regulation information, address information (address / zip code), facility information, telephone number information, and the like. The second map information 62 may be updated at any time by the communication device 20 communicating with another device.

The driving controller 80 includes, for example, an accelerator pedal, a brake pedal, a shift lever, a steering wheel, a deformed steering wheel, a joystick, and other controls. A sensor for detecting the amount of operation or the presence or absence of operation is attached to the operation controller 80, and the detection result is the automatic operation control device 100, or the traveling driving force output device 200, the brake device 210, and the steering device. It is output to a part or all of 220.

The automatic operation control device 100 includes, for example, a first control unit 120, a second control unit 160, and a processing unit 170. The first control unit 120, the second control unit 160, and the processing unit 170 are realized by, for example, a hardware processor such as a CPU (Central Processing Unit) executing a program (software). In addition, some or all of these components are hardware (circuits) such as LSI (Large Scale Integration), ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array), and GPU (Graphics Processing Unit). It may be realized by the part; including circuitry), or it may be realized by the cooperation of software and hardware. The program may be stored in advance in a storage device (a storage device including a non-transient storage medium) such as an HDD or a flash memory of the automatic operation control device 100, or may be detachable such as a DVD or a CD-ROM. It is stored in a storage medium, and may be installed in the HDD or flash memory of the automatic operation control device 100 by mounting the storage medium (non-transient storage medium) in the drive device.

The first control unit 120 includes, for example, a recognition unit 130 and an action plan generation unit 140. The first control unit 120, for example, realizes a function by AI (Artificial Intelligence) and a function by a model given in advance in parallel. For example, the function of "recognizing an intersection" is executed in parallel with the recognition of an intersection by deep learning or the like and the recognition based on a predetermined condition (there is a signal that can be pattern matched, a road marking, etc.). It may be realized by scoring and comprehensively evaluating. This ensures the reliability of autonomous driving.

The recognition unit 130 recognizes the position, speed, acceleration, and other states of objects around the vehicle M based on the information input via the object recognition device 16. The position of the object is recognized as, for example, a position on absolute coordinates with the representative point of the vehicle M (center of gravity, center of drive axis, etc.) as the origin, and is used for control. The "state" of an object may include the object's acceleration, jerk, or "behavioral state" (eg, whether it is changing lanes or is about to change lanes).

In principle, the action plan generation unit 140 travels in the recommended lane determined by the recommended lane determination unit 61, and the vehicle M automatically (operates by the driver) so as to be able to respond to the surrounding conditions of the vehicle M. Generate a target track to travel in the future (regardless of). The target trajectory contains, for example, a velocity element. For example, the target track is expressed as an arrangement of points (track points) to be reached by the vehicle M in order. The track point is a point to be reached by the vehicle M for each predetermined mileage (for example, about several [m]) along the road, and separately, a predetermined sampling time (for example, about 0 comma number [sec]). A target velocity and a target acceleration for each are generated as part of the target trajectory.

The action plan generation unit 140 may set an event for automatic driving when generating a target trajectory. Autonomous driving events include constant speed driving events, low speed following driving events, lane change events, branching events, merging events, takeover events, automatic parking events, and the like. The action plan generation unit 140 generates a target trajectory according to the activated event.

The automatic parking event is an event in which the vehicle M automatically parks in a predetermined parking position without depending on the operation of the occupant. The predetermined parking position may be a parking position designated by a parking lot management device (not shown), or may be a parkingable parking position (vacant parking position) recognized by the vehicle M. The vehicle M may execute an automatic parking event in cooperation with the parking lot management device or the terminal device 400 described above. For example, the vehicle M moves or parks in a designated direction or position based on the instruction transmitted by the parking lot management device. Further, the vehicle M may execute an automatic parking event based on the instruction of the terminal device 400 after the occupant gets off.

The second control unit 160 controls the traveling driving force output device 200, the brake device 210, and the steering device 220 so that the vehicle M passes the target track generated by the action plan generation unit 140 at the scheduled time. do.

The second control unit 160 acquires, for example, information on the target trajectory (orbit point) generated by the action plan generation unit 140 and stores it in a memory (not shown). The second control unit 160 controls the traveling driving force output device 200 or the brake device 210 based on the speed element associated with the target trajectory stored in the memory. The second control unit 160 controls the steering device 220 according to the degree of bending of the target trajectory stored in the memory.

The processing unit 170 generates information to be transmitted to the management device 500, and sets the destination of the vehicle M in cooperation with the agent device 300. Further, the processing unit 170 analyzes the image captured by the vehicle interior camera 310. The details of the processing of the processing unit 170 will be described later.

The traveling driving force output device 200 outputs a traveling driving force (torque) for the vehicle to travel to the drive wheels. The brake device 210 includes, for example, a brake caliper, a cylinder that transmits hydraulic pressure to the brake caliper, an electric motor that generates hydraulic pressure in the cylinder, and a brake ECU. The brake ECU controls the electric motor according to the information input from the second control unit 160 or the information input from the operation controller 80 so that the brake torque corresponding to the braking operation is output to each wheel. The steering device 220 includes, for example, a steering ECU and an electric motor. The electric motor, for example, exerts a force on the rack and pinion mechanism to change the direction of the steering wheel. The steering ECU drives the electric motor according to the information input from the second control unit 160 or the information input from the operation controller 80, and changes the direction of the steering wheel.

The agent device 300 interacts with the occupant of the vehicle M and provides services to the occupant. The service is information provision and task setting described later. The agent device 300 recognizes the voice of the occupant, selects the information to be provided to the occupant based on the recognition result, and causes the HMI 30 to output the selected information. Part or all of this function may be realized by AI (Artificial Intelligence) technology. Further, the agent device 300 may interact with an occupant or provide a service in cooperation with an agent server device (not shown) via the network NW.

The agent device 300 performs processing by, for example, a hardware processor such as a CPU executing a program (software). Some or all of the components included in the agent device 300 may be realized by hardware (circuit unit; including circuitry) such as LSI, ASIC, FPGA, GPU, or cooperation between software and hardware. May be realized by. The program may be stored in advance in a storage device such as an HDD or a flash memory (a storage device including a non-transient storage medium), or a removable storage medium (non-transient) such as a DVD or a CD-ROM. It is stored in a sex storage medium) and may be installed by mounting the storage medium in a drive device. The vehicle interior camera 310 is provided inside the vehicle of the vehicle M and is a camera that mainly captures the face of the user.

[Terminal device]
The terminal device 400 is, for example, a smartphone or a tablet terminal. The terminal device 400 is, for example, a terminal device held by an occupant (user) of the vehicle M. In the terminal device 400, an application program for using the service provided by the management system 1, a browser, or the like is activated to support the service described below. In the following description, it is assumed that the terminal device 400 is a smartphone and an application program (service application 410) for receiving a service is activated. The service application 410 communicates with the management device 500 to provide information to the user, and provides information based on the operation performed by the user to the terminal device 400 to the management device 500 or the terminal device 400.

[Management device]
FIG. 3 is a diagram showing an example of the functional configuration of the management device 500. The management device 500 includes, for example, a communication unit 502, an acquisition unit 504, an information generation unit 506, a provision unit 508, and a storage unit 520. The functional configuration in which the providing unit 508 or the information generation unit 506 and the providing unit 508 are combined is an example of the “providing unit”.

The communication unit 502 is, for example, a wireless communication module for connecting to a network NW or directly communicating with another terminal device or the like.

A part or all of the acquisition unit 504, the information generation unit 506, and the provision unit 508 are realized by, for example, a hardware processor such as a CPU executing a program (software). Some or all of these components may be realized by hardware such as LSI, ASIC, FPGA, GPU (circuit part; including circuitry), or realized by the cooperation of software and hardware. May be good. The program may be stored in advance in a storage device (a storage device including a non-transient storage medium) such as an HDD or a flash memory of the management device 500, or a removable storage medium such as a DVD or a CD-ROM. A storage medium (non-transient storage medium) may be attached to the drive device and installed in the HDD or flash memory of the management device 500. The storage unit 520 is realized by, for example, an HDD, a flash memory, an EEPROM (Electrically Erasable Programmable Read Only Memory), a ROM (Read Only Memory), a RAM (Random Access Memory), or the like.

For example, service information 522, map information 524, and the like are stored in the storage unit 520. The service information 522 is, for example, information acquired from the vehicle M and related to a service provided to the user. The service information 522 includes identification information of the user (guardian), a task to be executed by the robot device 600 designated by the guardian, an image of the guardian, an image of the target person, and a place where the vehicle M and the robot device 600 meet. , Time, password set by the guardian, etc. (see FIGS. 7 and 9). The map information 524 is, for example, information in which a road shape is expressed by a link indicating a road and a node connected by the link.

A guardian is a person who supports this movement when the target person heads for a predetermined destination. The guardian is, for example, the parent of the subject if the subject is a child. Further, the guardian is not limited to the parent-child relationship, but may simply be a person who supports the target person.

The acquisition unit 504 acquires the information provided by the vehicle M or the robot device 600. The acquired information is stored in the storage unit 520. The acquisition unit 504, for example, identifies the target person to be protected by the guardian from the vehicle M or the terminal device 400, and confirmation information for confirming that the target person has arrived at the meeting place with the guardian. And get. The identification information is, for example, a feature amount extracted from an image captured by the subject, and the confirmation information is, for example, a feature amount extracted from an image captured by a guardian. The feature amount is, for example, a distribution of a luminance value or an index indicating a change in luminance (for example, an index indicating a luminance gradient).

The information generation unit 506 provides the robot device 600 with instruction information and confirmation information including the identification information for guiding the target person from the meeting place to the destination based on the identification information and the confirmation information. .. The providing unit 508 provides the generated instruction information and confirmation information to the robot device 600. Details of these will be described later.

[Robot device]
FIG. 4 is a diagram showing an example of the functional configuration of the robot device 600. The robot device 600 includes, for example, a communication unit 602, a camera 604, a touch panel 606, a position specifying unit 608, a drive unit 610, a drive control unit 612, an information management unit 614, a specific unit 616, and a control unit. 618 and a storage unit 630 are provided. A part or all of the drive control unit 612, the information management unit 614, the specific unit 616, and the control unit 618 are realized by, for example, a hardware processor such as a CPU executing a program (software). Some or all of these components may be realized by hardware such as LSI, ASIC, FPGA, GPU (circuit part; including circuitry), or realized by the cooperation of software and hardware. May be good. The program may be stored in advance in a storage device (a storage device including a non-transient storage medium) such as an HDD or a flash memory of the robot device 600, or a removable storage medium such as a DVD or a CD-ROM. A storage medium (non-transient storage medium) may be attached to the drive device and installed in the HDD or flash memory of the robot device 600. The storage unit 630 is realized by, for example, an HDD, a flash memory, an EEPROM, a ROM, a RAM, or the like.

The information provided by the management device 500 is stored in the storage unit 630. For example, task management information 632 and map information 634 are stored in the storage unit 630. The task management information 632 is information necessary for the robot device 600 to execute a task, such as information for identifying a guardian and a target person and the contents of the task (FIG. 9 described later). The task management information 632 is an example of "instruction information" and "confirmation information".

The communication unit 602 is, for example, a wireless communication module for connecting to a network NW or directly communicating with another terminal device or the like. The communication unit 602 performs wireless communication based on DSRC, Bluetooth, and other communication standards.

The camera 604 is, for example, a digital camera using a solid-state image sensor such as a CCD or CMOS. The camera 604 is attached to any part of the robot device 600. The camera 604 is attached to a position where a person existing around the robot device 600 can be imaged.

The touch panel 606 is a device in which a display device and an input device are combined. The user performs a touch operation, a swipe operation, or the like on the image displayed on the display device to select information or input information.

The position specifying unit 608 determines the position of its own machine based on radio waves arriving from, for example, a GNSS satellite (for example, a GPS satellite).

The drive unit 610 includes, for example, a drive source such as a motor, a transmission mechanism for transmitting the power generated by driving the drive source, and the like. The traveling unit (for example, wheels) is operated by the power transmitted by the driving unit 610 to drive the robot device 600. For example, when the drive source is a motor, the robot device 600 includes a battery that supplies electric power to the motor. The drive control unit 612 controls a drive source such as a motor. The robot device 600 may be a bipedal walking type robot.

The information management unit 614 manages the information acquired from the management device 500. The information management unit 614 acquires, for example, the information transmitted by the management device 500, and stores the acquired information in the storage unit 630.

The identification unit 616 uses the information managed by the information management unit 614 to specify the target person to be guided by the robot device 600, and the guardian places the target person at the meeting place with the robot device 600. Identify what you brought. The specifying unit 616 identifies the target person to be guided based on the feature amount included in the task management information 632 provided by the management device 500 and the feature amount obtained from the image captured by the camera 604. When the information indicating the characteristics of the person included in the image captured by the camera 604 and the feature information included in the task management information 632 match, the specific unit 616 targets the person captured by the camera 604 to be guided. Identify as a person. The match is not limited to an exact match, and may be a match of a predetermined degree or more. Similarly, the specifying unit 616 identifies that the guardian has brought the target person to the meeting place by using the feature amount of the guardian included in the task management information 632.

The control unit 618 controls the robot device 600 to guide the target person to the destination based on the task management information 632. The control unit 618 makes the robot device 600 stand by at a predetermined meeting place, or moves the robot device 600 to the destination while guiding the target person. Further, the control unit 618 displays information on the display unit of the touch panel 606, and outputs voice to a speaker (not shown).

[Overview of services provided by the management system]
FIG. 5 is a diagram for explaining an outline of the service provided by the management system 1.
(1) The vehicle M transmits, for example, an image of a guardian and a target person acquired in the vehicle interior, a password, a task, an estimated time of arrival to arrive at the meeting place, and the like to the management device 500.
(2) The management device 500 transmits the information acquired from the vehicle M (or the information obtained by processing the acquired information) to the robot device 600 that guides the target person.
(3) The robot device 600 guides and guards the target person and guides the target person to the destination.
(4) When the target person has an abnormality while guiding the target person to the destination, the robot device 600 transmits information indicating that the abnormality has occurred to the management device 500. Then, the management device 500 transmits information indicating that an abnormality has occurred in the vehicle M or the terminal device 400.

[Processes performed by the vehicle]
FIG. 6 is a diagram showing an example of a processing flow executed by the processing unit 170 of the vehicle M. First, the processing unit 170 sets a task based on the operation of the guardian who is the occupant of the vehicle M with respect to the HMI 30 (step S100). The task is a task to be executed by the robot device 600. The task is, for example, to have the robot device 600 guide the target person from the point A to the point B, which is the meeting place.

Next, the processing unit 170 sets a password based on the operation of the guardian who is the occupant of the vehicle M on the HMI 30 (step S102). Next, the processing unit 170 acquires images of the guardian and the target person based on or automatically by the operation of the guardian who is the occupant of the vehicle M with respect to the HMI 30 (step S104). The image is an image taken by the vehicle interior camera 310.

Next, the processing unit 170 acquires the estimated time of arrival when the vehicle M is scheduled to arrive at the meeting place (step S106). The estimated time of arrival is, for example, the time estimated by the navigation device 50. Next, the processing unit 170 associates the information acquired in steps S100 to S106 described above, and transmits a request including the associated information to the management device 500 (step S108). This ends one routine in this flowchart.

FIG. 7 is a diagram showing an example of the image IM1 displayed on the HMI 30 of the vehicle M. The guardian inputs each information by performing a predetermined operation on the HMI 30. The image IM1 is an image for confirming the information input by the guardian, including the input information and the like. The image IM1 includes, for example, a set task, a set password, a meeting place, an estimated arrival time at the meeting place, an image of a guardian, an image of a target person, and the like. Further, when the guidance route confirmation button of the image IM1 is operated, information indicating the route from the meeting place to the destination is displayed on the HMI 30. When the setting button of the image IM1 is operated, a request including the information displayed on the image IM1 (a request including the associated information) is transmitted to the management device 500.

The setting of the above task can be performed by, for example, a preset person. For example, the management device 500 may allow the task setting when the preset password is input, or may allow the task setting when the feature amount of the preset image of the person is acquired. You may. The feature amount of the image of the person is, for example, the feature amount obtained from the image captured by the vehicle M. Further, the end of the task (for example, cancellation of the task or stopping the guidance by the robot device 600 while the robot device 600 is guiding the target person) can be performed only by a preset person. May be good.

As described above, the guardian can easily set the task and easily confirm the content of the set task.

Further, as described above, when the task is set, the automatic operation control device 100 sets the meeting place as the destination and automatically drives toward the set destination. This improves the convenience of parents.

[Processes executed by the management device]
FIG. 8 is a flowchart showing an example of the flow of processing executed by the management device 500. First, the management device 500 determines whether or not the request has been received (step S200). When the request is received, the management device 500 extracts the robot device 600 based on the received request (step S202). For example, the management device 500 extracts a robot device 600 capable of providing a service from a plurality of robot devices 600 to be managed.

Next, the management device 500 extracts the feature amount from the images of the guardian and the target person acquired from the vehicle M (step S204). The management device 500 extracts the feature amount of the target person from the image of the target person, and extracts the feature amount of the guardian from the image of the guardian.

Next, the management device 500 generates request information using the information acquired from the vehicle M and the above-mentioned feature amount (step S206), and transmits the generated request information to the robot device 600 that provides the service (step S206). Step S208). The request information is information indicating a request to guide the target person corresponding to the guardian who sent the request to the robot device 600 to the destination. This ends one routine in this flowchart.

FIG. 9 is a diagram showing an example of information acquired from the vehicle M by the management device 500 (information before processing, service information 522) and information transmitted to the robot device 600 (information after processing, task management information 632). Is. The management device 500 transmits the task management information 632, which is the processed information obtained from the vehicle M, to the robot device 600. For example, the management device 500 associates the feature amount of the guardian and the feature amount of the target person with the task, the password, the estimated arrival time, the meeting place, the route, etc., instead of the image of the guardian and the image of the target person. , The associated information is transmitted to the robot device 600.

As described above, the management device 500 processes the information acquired from the vehicle M into the information necessary for the robot device 600 to execute the task, and provides the processed information (instruction information) to the robot device 600. As a result, the robot device 600 reduces the burden of processing information.

In the above example, the management device 500 has been described as extracting the feature amount from the image, but the management device 500 may provide the image to the robot device 600, and the robot device 600 may extract the feature amount. Further, the vehicle M may extract a feature amount from the image and provide the extracted feature amount to the management device 500.

Further, in the above example, when the management device 500 extracts the robot device 600 that guides the target person and requests the robot device 600 to perform a task, the image of the robot device 600 and information (colors, etc.) indicating the features are shown. Features such as shape) may be provided to the vehicle M or the terminal device 400. The image and the information indicating the feature transmitted by the management device 500 are displayed on the display unit of the vehicle M or the terminal device 400. By confirming this information, the guardian or the target person can easily recognize the robot device 600 that performs the task at the meeting place.

[Processes executed by the robot device]
FIG. 10 is a flowchart showing an example of the flow of processing executed by the robot device 600. The robot device 600 stands by at the meeting place at the estimated arrival time. The estimated arrival time may be updated as appropriate. The estimated arrival time is updated by the position information of the vehicle M and the estimated arrival time re-estimated in the vehicle M. The following is the process executed after the guardian and the target person arrive at the meeting place.

First, the robot device 600 determines whether or not a password has been input to the touch panel 606 of the robot device (step S300). When the password is input, the robot device 600 determines whether or not the input password matches the password transmitted by the management device 500 (step S302).

If the passwords match, the robot device 600 determines whether or not the image of the target person and the image of the guardian have been captured (step S304). For example, when a person approaches, the robot device 600 announces to the person to enter a password or bring the face closer to a position for capturing an image. When the face approaches a predetermined position, the robot device 600 automatically captures an image.

When the image is captured, the robot device 600 matches the feature amount of the image of the target person with the feature amount of the image of the target person transmitted by the management device 500, and further manages the feature amount of the image of the guardian. It is determined whether or not the feature amount of the guardian's image transmitted by the device 500 matches (step S305). When it is determined that these match, the robot device 600 starts a process of guiding the target person to the destination (step S308). The fact that the feature amount of the image of the target person and the feature amount of the image of the target person transmitted by the management device 500 match means that "the identification information input by the target person and the identification information provided by the providing unit match". This is an example of that. Matching the feature amount of the guardian's image with the feature amount of the guardian's image transmitted by the management device 500 is an example of "the processing based on the confirmation information is completed".

This ends one routine in this flowchart. If a negative determination result is obtained in step S302, step S304, or step S306, one routine in this flowchart ends.

As described above, the management device 500 provides the robot device 600 with information for authenticating that the guardian and the target person have arrived at the meeting place. Then, the robot device 600 identifies the target person brought by the guardian based on the provided information, and guides the target person to the destination when the target person can be identified. As a result, the robot device 600 can more reliably confirm that the target person has been brought to the guardian and can surely guide the target person to the destination.

Further, as described above, the robot device 600 can more reliably determine whether the guardian has brought the target person to the meeting place by making a double judgment based on the password and the image.

[Processes performed by robot devices, management devices, and vehicles]
FIG. 11 is a flowchart showing an example of a flow of processing executed by a robot device, a management device, and a vehicle. This process is a process executed after the guidance is started.

First, the robot device 600 acquires an image (or moving image) of the target person during guidance (step S400). Next, the robot device 600 transmits the acquired image to the management device 500 (step S402). The management device 500 transmits the acquired image to the vehicle M (or the terminal device 400) (step S500). As a result, an image showing the state of the target person is displayed on the display unit of the vehicle M, and the guardian can confirm the state of the target person in the vehicle M, so that the guardian's sense of security is improved.

Next, the robot device 600 analyzes the image captured in step S402 (step S406), and determines whether or not an abnormality has occurred based on the analysis result (step S406). Abnormalities include accidents, disasters, crimes, and the fact that the target person and the robot device 600 are separated from each other. For example, the robot device 600 uses a neural network stored in the storage unit to determine whether or not the above abnormality has occurred, or uses pattern information stored in the storage unit to determine based on a pattern matching method. It may be determined whether or not the above abnormality has occurred.

When an abnormality occurs, the robot device 600 notifies the management device 500 that the abnormality has occurred (step S408). When the management device 500 obtains the notification, the management device 500 notifies the vehicle M that an abnormality has occurred (step S502). Further, the management device 500 may notify a predetermined organization or a preset notification destination that an abnormality has occurred. Next, when the robot device 600 recognizes that an abnormality has occurred, it causes the output unit to output an alarm (step S410).

As described above, the robot device 600 notifies a predetermined notification destination of the occurrence of an abnormality or outputs an alarm so as not to cause a disadvantage to the target person. As a result, it is possible to prevent the target person from being disadvantaged, and further to improve the sense of security of the guardian.

Next, the management device 500 determines whether or not the robot device 600 has arrived at the destination (step S412). If the robot device 600 has not arrived at the destination, the process returns to the process of step S400. When the robot device 600 arrives at the destination, one routine of this flowchart ends.

The robot device 600 may store the image or voice being guided in the storage unit 630 and provide the stored information to the management device 500. For example, the management device 500 provides the vehicle M or the terminal device 400 with images and sounds being guided in response to a request from a guardian or a request from a predetermined organization. For example, when an accident occurs, the stored information is analyzed to confirm the cause of the accident.

As described above, the robot device 600, the management device 500, and the vehicle can cooperate with each other to watch over the target person until the target person arrives at the destination. It is possible to improve the sense of security of parents.

In the above example, when the robot device 600 detects an abnormality or failure of its own device, the robot device 600 may provide this information to the management device 500, the vehicle M, or the terminal device 400.

Further, when the robot device 600 arrives at the destination, when a person waiting at the destination inputs a predetermined password, or when the feature amount obtained from the image of the person matches the preset feature amount. In this case, information (completion information) indicating that the vehicle has arrived at the destination may be transmitted to the management device 500, the vehicle M, or the terminal device 400. This allows the guardian to recognize that the subject has arrived at the destination. Further, the information indicating that the user has arrived at the destination may be transmitted to a designated terminal device 400 or vehicle M different from the terminal device 400 or vehicle M that sent the request.

According to the first embodiment described above, the management device 500 processes the information acquired from the vehicle M or the terminal device 400 for guiding the target person from the meeting place to the destination to the robot device 600. By providing the robot device, it is possible to reliably guide the target person taken by the guardian to the destination.

<Second Embodiment>
Hereinafter, the second embodiment will be described. In the first embodiment, the feature amount obtained from the image of the guardian is provided to the robot device 600. In the second embodiment, the position information (confirmation information) of the vehicle M is transmitted to the robot device 600 in place of (or in addition to) the feature amount obtained from the image of the guardian. Hereinafter, the second embodiment will be described.

In the second embodiment, when the vehicle M that has transmitted the request arrives at the meeting place (within a predetermined range from the meeting place), the management device 500 estimates that the guardian has arrived at the meeting place.

FIG. 12 is a flowchart showing an example of the flow of processing executed by the robot device 600 of the second embodiment. The robot device 600 stands by at the meeting place by the estimated arrival time.

First, it is determined whether or not the robot device 600 has acquired the information indicating that the vehicle M that has transmitted the request from the management device 500 has arrived at the meeting place (step S600). The management device 500 acquires the position information of the vehicle M at predetermined intervals, and when the acquired position information approaches the meeting place or the position information matches the position information corresponding to the meeting place, the management device 500 transmits the request. It is estimated that M has arrived at the meeting place. Acquiring the information indicating that the maintenance vehicle M has arrived at the meeting place is an example of "the processing based on the confirmation information is completed".

When the robot device 600 acquires the information indicating that the vehicle M that has transmitted the request has arrived at the meeting place, the robot device 600 determines whether or not the image of the target person has been captured (step S602). When the image is captured, the robot device 600 determines whether or not the feature amount of the image of the target person matches the feature amount of the image of the target person transmitted by the management device 500 (step S604). If it is determined that the robot device 600 matches, the robot device 600 starts a process of guiding the target person to the destination (step S606). This ends one routine in this flowchart. If a negative determination result is obtained in step S602 or step S604, one routine in this flowchart ends.

As described above, the management device 500 can make the robot device 600 more easily recognize that the guardian has brought the target person to the meeting place by using the position information of the vehicle M.

According to the second embodiment described above, the management system 1 can more easily and surely guide the target person taken by the guardian to the destination.

In each of the above embodiments, an example in which an image is used when the robot device 600 confirms a guardian or a target person has been described, but instead (in addition), other identification information is used. You may. The other identification information may be a password, or may be unique information possessed by a guardian or a target person. The unique information includes fingerprint information of a guardian or a target person, voiceprint information (characteristic information indicating the characteristics of voice), iris information, and the like. Other identification information is an example of "identification information of the subject" or "confirmation information".

A part or all of the functional configurations included in the management device 500 may be included in the robot apparatus 600, and a part of the functional configurations included in the robot apparatus 600 may be included in the management apparatus 500. For example, the entire functional configuration of the management device 500 may be included in the robot device 600, and the above-mentioned processing in which the vehicle M and the robot device 600 cooperate may be executed. Further, a part of the functional configuration of the management device 500 or the robot device 600 may be included in the vehicle M, and a part of the functional configuration of the vehicle M may be included in the management device 500 or the robot device 600. ..

Further, in the above example, the guardian sends a request in the vehicle M or goes to the meeting place, but instead (or in addition), another means of transportation is used to reach the meeting place. You may head for it. In this case, the guardian inputs a request using the terminal device 400 or sends the request to the management device 500.

Although the embodiments for carrying out the present invention have been described above using the embodiments, the present invention is not limited to these embodiments, and various modifications and substitutions are made without departing from the gist of the present invention. Can be added.

1 Management system, 2 Vehicle system, 300 Agent device, 400 Terminal device, 410 Service application, 500 Management device, 502 Communication unit, 504 Acquisition unit, 506 Information generation unit, 508 Provision unit 520 ... Storage unit, 522 ... Service information, 600 ... Robot device, 632 ... Task management information

Claims (15)

It is a management device that manages robot devices.
An acquisition unit that acquires identification information of the target person protected by the guardian from the vehicle or terminal device and confirmation information for confirming that the target person has arrived at the meeting place with the guardian.
Based on the identification information and the confirmation information acquired by the acquisition unit, the instruction information including the identification information and the confirmation information for guiding the target person from the meeting place to the destination to the robot device are provided. , The providing unit provided to the robot device,
A management device equipped with. The confirmation information includes an image captured by the guardian, feature information indicating features extracted from the image captured by the guardian, feature information indicating the characteristics of the voice of the guardian, and position information of the vehicle. The location information of the terminal device, a preset password,
The management device according to claim 1. The subject identification information includes an image captured by the subject, feature information indicating features extracted from the image captured by the subject, feature information indicating the characteristics of the voice of the subject, and the subject. The location information of the device held by, or the preset password,
The management device according to claim 1 or 2. The instruction information includes an instruction to make the robot device stand by at a time scheduled to arrive near the meeting place and to guide the target person to a destination after the target person arrives at the meeting place.
The management device according to any one of claims 1 to 3. When the providing unit acquires the completion information from the robot device indicating that the robot device has guided the target person to the destination, the providing unit provides the completion information to the vehicle or the terminal device.
The management device according to any one of claims 1 to 4. The providing unit provides information indicating the characteristics of the robot device that guides the target person to the destination to the vehicle or the terminal device.
The management device according to any one of claims 1 to 5. The management device according to any one of claims 1 to 6 and the management device.
A robot device that guides the target person from the meeting place to the destination based on the instruction information provided by the management device providing unit.
Management system equipped with. The robot device is
The instruction information and the confirmation information including the identification information are acquired from the management device, and the instruction information and the confirmation information are acquired.
When the process based on the confirmation information is completed and the identification information input by the target person arriving at the meeting place and the identification information provided by the providing unit match, the target person is referred from the meeting place to the purpose. Guide to the ground,
The management system according to claim 7. The robot device is
The estimated time of arrival of the target person is acquired from the vehicle or the terminal device, and the person waits at the meeting place at the estimated time of arrival.
The management system according to claim 7 or 8. The robot device is
While guiding the subject from the meeting place to the destination, the image captured by the image pickup device mounted on the robot device is analyzed to determine whether or not an abnormality has occurred in the subject. Judgment,
When it is determined that the abnormality has occurred, information indicating that the abnormality has occurred is provided to the vehicle or the terminal device.
The management system according to any one of claims 7 to 9. The robot device is
While guiding the target person from the meeting place to the destination, the image captured by the image pickup device mounted on the robot device is analyzed to determine whether or not an abnormality has occurred in the target person. Judgment,
When it is determined that the abnormality has occurred, information indicating that the abnormality has occurred is output to the output unit mounted on the robot device.
The management system according to any one of claims 7 to 10. Further, the vehicle that transmits the identification information of the target person to be protected by the guardian and the confirmation information for confirming that the target person has arrived at the meeting place with the guardian to the management device. include,
The management system according to any one of claims 7 to 11. The management device according to any one of claims 1 to 6 and the management device.
A vehicle that transmits identification information of a target person to be protected by the guardian and confirmation information for confirming that the target person has arrived at the meeting place with the guardian, and a vehicle.
Management system equipped with. It is a management method for managing robot devices.
The computer
The identification information of the target person to be protected by the guardian and the confirmation information indicating that the target person has arrived at the meeting place with the guardian are obtained from the vehicle or the terminal device.
Based on the acquired identification information and confirmation information, the instruction information including the identification information and the confirmation information for guiding the target person from the meeting place to the destination to the robot device are described. Provide to robot equipment,
Management method. A program for managing robot devices,
On the computer
The vehicle or the terminal device is made to acquire the identification information of the target person to be protected by the guardian and the confirmation information indicating that the target person has arrived at the meeting place by the guardian.
Based on the acquired identification information and confirmation information, the instruction information including the identification information and the confirmation information for guiding the target person from the meeting place to the destination to the robot device are described. Have the robot device provide it,
program.

Images