JP7273638B2 - ROBOT MANAGEMENT DEVICE, ROBOT MANAGEMENT METHOD AND ROBOT MANAGEMENT SYSTEM - Google Patents

Description

The present invention relates to a robot management device, a robot management method, and a robot management system using these.

In recent years, as one means of resolving the labor shortage, various techniques related to robot control systems using guide robots have been proposed (see, for example, Patent Document 1). In the robot control system described in Patent Literature 1, a guide robot (communication robot) installed in a facility and a sensor installed in the facility are linked to enable appropriate customer service to visitors.

Japanese Patent No. 6142306

However, when identifying a visitor using a sensor installed in a facility, it may be difficult to identify the visitor depending on the position and posture of the visitor. For example, when a visitor visits while talking, it is difficult to identify the visitor when the direction of walking is different from the direction of the face. If it becomes difficult to identify a visitor, it becomes difficult to serve customers efficiently using a guidance robot.

One aspect of the present invention is a robot management device that manages a self-propelled guide robot that provides services to visitors visiting a facility. The robot management device includes an in-facility image acquisition unit that acquires an in-facility image captured by an imaging unit installed in the facility, and identifies visitors who have visited the facility from the in-facility image acquired by the in-facility image acquisition unit. A visitor identification unit, a visitor group identification unit that identifies a visitor group that has visited the facility from the image of the facility acquired by the image acquisition unit of the facility, and a guidance robot that dispatches a guidance robot under the visitor and a robot movement instruction unit that instructs the movement of the visitor, and when the visitor identification unit determines that the visitor cannot be identified based on the in-facility image, the robot movement instruction unit determines that the visitor is unidentifiable. Instruct the guide robot to move so that the guide robot is dispatched under the visitor, and the visitor who is determined to be unidentifiable by the visitor identification unit in the visitor group identified by the visitor group identification unit exists, the guide robot is instructed to move so that the guide robot is dispatched under the visitor group to which the visitor determined to be unidentifiable belongs.

Another aspect of the present invention is a robot management method for managing a self-propelled guide robot that provides services to visitors visiting a facility. The robot management method includes an in-facility image acquisition step of acquiring an in-facility image captured by an imaging unit installed in the facility, and an in-facility image acquired in the in-facility image acquisition step, which is executed by a computer. a visitor group identification step of identifying a visitor who has visited the facility, a visitor group identification step of identifying a visitor group who has visited the facility from the image of the facility acquired in the facility image acquisition step, and guidance under the visitor and a robot movement instruction step for instructing the guide robot to move so that the robot is dispatched. In the step of instructing movement of the robot, when it is determined that the visitor cannot be specified based on the image inside the facility in the step of specifying the visitor, a guide robot is dispatched to the visitor who is determined as being unspecifiable. If the visitor group identified in the visitor group identification step includes a visitor determined to be unidentifiable by the visitor identification step after instructing the guide robot to move, the visitor is determined to be unidentifiable. The guide robot is instructed to move so that the guide robot is dispatched under the visitor group to which the visitor belongs .

Further, a robot management system according to another aspect of the present invention includes the above-described robot management device, an imaging unit installed in a facility and capable of communicating with the robot management device, and a self-propelled robot management device installed in the facility. a communicable guide robot;

ADVANTAGE OF THE INVENTION According to this invention, the efficient customer service using a guide robot can be appropriately performed with respect to the visitor who visited the facility.

1 is a schematic configuration diagram of a robot management system using a server device that constitutes a robot management device according to an embodiment of the present invention; FIG. FIG. 2 is a perspective view schematically showing a guide robot that constitutes the robot management system shown in FIG. 1; FIG. 2 is a block diagram showing the main configuration of the robot management system shown in FIG. 1; FIG. 4 is a block diagram showing the main configuration of the server device shown in FIG. 3; FIG. 4 is a flowchart showing an example of a guide robot dispatching process executed by a computing unit of the server device of FIG. 3; FIG. FIG. 4 is a flowchart showing an example of visitor identification processing executed by a computing unit of the server device of FIG. 3; FIG. FIG. 4 is a flowchart showing an example of visitor group identification processing executed by a computing unit of the server device of FIG. 3; FIG.

An embodiment of the present invention will be described below with reference to FIGS. 1 to 7. FIG. A robot management device according to an embodiment of the present invention identifies a visitor who has visited a facility based on an in-facility image captured by an imaging unit installed in the facility. For example, the visitor is identified based on the face image of the visitor appearing in the image inside the facility.

At this time, if the visitor cannot be identified based on the in-facility image, a guide robot is dispatched to the visitor who is determined to be unidentifiable. For example, when the face image of the visitor cannot be extracted (recognized) from the image inside the facility, the visitor is determined to be unidentifiable, and a guidance robot is dispatched to the visitor determined to be unidentifiable.

In this way, by dispatching the guide robot to the visitor who is determined to be unidentifiable, the guide robot can detect changes in the position or posture of the visitor so that the visitor can be identified by the image of the facility. It is possible to prompt the visitor, or to have the guide robot directly capture the face image of the visitor. For example, when the guide robot moves and the visitor turns to face the guide robot, it becomes easier for the imaging unit to capture an image of the visitor's face, and the guide robot can directly capture an image of the visitor's face. This makes it possible to identify the visitor who has been determined to be unidentifiable. As a result, for example, it is possible to provide appropriate customer service through cooperation between the imaging unit and the guide robot.

For example, it is possible to provide a service in which a visitor who is in a bad mood is sent a guide robot to communicate with the visitor. For example, by dispatching a guide robot to a visitor who has been kept waiting for a long time and is in a bad mood to communicate with the visitor, it is possible to make the visitor feel better or to prevent the visitor from being in a bad mood. As a result, it is possible to smoothly serve customers by staff members of the facility after that.

Also, for example, it is possible to have the guide robot ask the visitor about their requirements in advance or provide simple guidance. For example, for a visitor who has been waiting for a long time or a group of visitors who are expected to wait for a long time, the guide robot can be made to ask requirements and the like or provide simple guidance in advance. This enables efficient customer service by facility staff and the like thereafter.

Thus, by using the robot management device, efficient and smooth customer service using the guide robot becomes possible.

In the robot management system of this embodiment, the robot management device described above, an imaging unit such as a camera installed in the facility and capable of communicating with the robot management device, A robot-dispatched guidance service as described above is provided using the possible guidance robots described above. In this way, by providing a robot dispatch guide service using a robot management system, for example, efficient and smooth customer service using a guide robot becomes possible.

As a business entity that provides the robot dispatch guide service as described above, for example, retail stores that sell various products, art museums, museums, galleries and other art related facilities such as galleries, science museums, memorial halls, exhibitions, workshops, etc. shops and the like. Stores that retail various products include, for example, department stores, supermarkets, specialty stores, and the like. Examples of specialty stores include various specialty stores and car dealers. In the following embodiments, the robot management device is configured by a server device, this server device is installed in an automobile dealership, and a guide robot arranged in the showroom is placed under a visitor who visits the showroom of the automobile dealership. An example of dispatching will be explained.

FIG. 1 is a schematic configuration diagram of a robot management system 100 using a server device 4 that constitutes a robot management device according to an embodiment of the present invention. As shown in FIG. 1, in a robot management system 100 using a server device 4 according to this embodiment, a self-propelled guide robot 3 is arranged in a showroom 1 of an automobile dealership where exhibition vehicles 2 are exhibited. A plurality of imaging units 11 installed on the ceiling 10 of the showroom 1 captures an image of a visitor visiting the showroom 1, and the visitor is identified from the captured image of the inside of the showroom.

For example, as shown in FIG. 1, when a friend group 101 consisting of three visitors A, B, and C and a parent-child group 102 consisting of two visitors D and E visit showroom 1, The server device 4 identifies visitors A to C from the in-showroom image including the friend group 101 captured by the imaging unit 11, and identifies visitors D, C from the in-showroom image including the parent-child group 102 captured by the imaging unit 11. Identify E. The server device 4 extracts (recognizes the faces of) the facial images of the visitors A to E from these in-showroom images, and identifies the visitors A to E based on the facial images.

At this time, if the visitors A to E cannot be identified based on these in-showroom images, the server device 4 dispatches the guide robot 3 to the visitors A to E determined as unidentifiable. For example, if the visitor E (for example, a child) in the parent-child group 102 cannot be specified, the server device 4 instructs the guide robot 3 to dispatch the guide robot 3 to the visitor E (for example, a child). Instruct the visitor E to move down. By moving the guide robot 3 under the visitor E, for example, the visitor E notices the guide robot 3, and by turning toward the guide robot 3, the position and posture of the visitor E can be changed.

Here, the guide robot 3 will be briefly described with reference to FIG. FIG. 2 is a perspective view schematically showing the guide robot 3 that constitutes the robot management system 100 shown in FIG. As shown in FIG. 2, the guide robot 3 is formed in an upright, approximately gourd shape, with a constricted portion 302 at the center and a head portion 301 above it and a body portion 303 below it. The guide robot 3 has a head 301 slightly larger than the torso 303, and has a shape of approximately two heads, and has a charming and warm shape as a whole. In this embodiment, the guide robot 3 has a height of around 110 cm.

Further, the guide robot 3 has no hands and feet, and is configured to be movable in any direction of 360 degrees, such as front, back, left, right, oblique directions, etc., by a traveling device 304 provided at the lower end of the body portion 303 . Note that the description of the specific configuration of the travel device 304 is omitted here. In this way, the guide robot 3 has a shape in which the head 301 is slightly larger than the body 303 and has no limbs, so that it is easy for a child to hug and communicate with the child. In addition, since the guide robot 3 can move while the traveling device 304 swings in the front-rear direction and the left-right direction, it is possible for the visitor to easily notice the approach and to facilitate communication with the visitor. there is

A head portion 301 of the guide robot 3 is provided with a horizontally elongated substantially elliptical face portion 305. The face portion 305 is configured to be able to display the expression of the guide robot 3, a simple character image, and the like. In this embodiment, a pair of pseudo eyes 306, 306 representing eyes are displayed on the face portion 305 of the guide robot 3, and the pair of pseudo eyes 306, 306 are configured to be able to express various facial expressions. For example, by changing the shape of the pair of pseudo eyes 306, 306, facial expressions such as emotions can be expressed. In this embodiment, a pseudo mouth 307 expressing a mouth is also displayed, and by changing the shape of the pair of pseudo eyes 306 and 306 and the pseudo mouth 307, the change in facial expression is made easier to understand.

Further, the guide robot 3 is configured to be able to move the positions of the pair of pseudo eyes 306 , 306 within the face 305 . The guide robot 3 changes the positions of the pair of pseudo eyes 306 and 306 within the face 305 to express the action of moving the line of sight. The guide robot 3 guides the visitor's line of sight by changing the positions of the pair of pseudo eyes 306 in front of the visitor to express movement of the line of sight. At this time, by causing the guide robot 3 to rotate in conjunction with the movement of the line of sight, it becomes easier to guide the line of sight of the visitor.

By dispatching the guidance robot 3 formed as described above to, for example, an unidentifiable visitor E, the position and position of the visitor E can be determined so that the face of the visitor E can be imaged by the imaging unit 11, for example. It is possible to cause the guide robot 3 to move to change its posture (movement to guide), or to directly image the face of the visitor E with the guide robot 3 . As a result, for example, the visitor E can be specified, and efficient and smooth customer service using the guide robot is possible. In order to satisfactorily provide such a robot dispatch guide service by car dealers, in the present embodiment, a robot management system 100 using the following server device 4 is configured.

FIG. 3 is a block diagram showing the essential configuration of the robot management system 100 shown in FIG. 1, and FIG. 4 is a block diagram showing the essential configuration of the server device 4 shown in FIG. As shown in FIG. 1, a robot management system 100 according to the present embodiment includes an imaging unit 11 installed on the ceiling 10 of a showroom 1 of a car dealership, and a self-propelled guide robot 3 installed in the showroom 1. and a server device 4 owned by a car dealer.

As shown in FIG. 3, the imaging unit 11, the guide robot 3, and the server device 4 are connected to a communication network 5 such as a wireless communication network, the Internet, or a telephone line network. Although FIG. 3 shows only one imaging unit 11 for the sake of convenience, there are actually a plurality of imaging units 11 as shown in FIG. Similarly, although only one guide robot 3 is shown in FIG. 3, multiple guide robots 3 can be arranged. If a large number of guide robots 3 are arranged in the showroom 1, the visitor may feel intimidated and the cost of the car dealer increases.

As shown in FIG. 3 , the imaging section 11 has a communication section 111 , a camera section 112 , a sensor section 113 , a storage section 114 and a calculation section 115 . The communication unit 111 is configured to be able to wirelessly communicate with the server device 4 and the guide robot 3 via the communication network 5 . The camera unit 112 is a camera having an imaging device such as a CCD or CMOS, and is configured to be able to capture an image of a visitor visiting the showroom 1 . The sensor unit 113 is a sensor such as a moving body detection sensor or a human sensor, and is configured to be able to detect the position and movement of a visitor who has visited the showroom 1 within the showroom. A plurality of camera units 112 and sensor units 113 are arranged on the ceiling 10 of the showroom 1 so that a visitor who visits the showroom 1 can be imaged and detected at any position within the showroom 1 .

Storage unit 114 has a volatile or non-volatile memory (not shown). The storage unit 114 stores various programs and data executed by the calculation unit 115 . For example, the storage unit 114 temporarily stores an image of the inside of the showroom captured by the camera unit 112 and the position information of the visitor detected by the sensor unit 113 .

The calculation unit 115 has a CPU and executes predetermined processing based on signals received from outside the imaging unit 11 via the communication unit 111 and various programs stored in the storage unit 114. A predetermined control signal is output to the unit 112 , the sensor unit 113 and the storage unit 114 .

For example, the calculation unit 115 outputs a control signal for causing the camera unit 112 to capture an image of the showroom 1 based on the signal received from the server device 4 via the communication unit 111 and a predetermined program, and outputs a control signal to the sensor unit 113 to capture the image of the visitor. Outputs a control signal to detect the position. Then, the calculation unit 115 outputs a control signal for transmitting the captured image of the inside of the showroom and the detected position information of the visitor to the server device 4 via the communication unit 111 . By this processing in the calculation unit 115 , the inside of the showroom 1 is imaged, the position of the visitor is detected, and the image inside the showroom and the position information of the visitor are transmitted to the server device 4 .

As shown in FIG. 3 , the guide robot 3 includes a communication section 31 , an input section 32 , an output section 33 , a camera section 34 , a traveling section 35 , a sensor section 36 and a storage section 37 as functional components. , and a calculation unit 38 . The communication unit 31 is configured to be able to wirelessly communicate with the server device 4 and the imaging unit 11 via the communication network 5 . The input unit 32 has various switch buttons (not shown) that can be operated during maintenance and the like, a microphone (not shown) that can input voices of visitors, and the like.

The output unit 33 has a speaker (not shown) capable of outputting audio, and a display unit 331 capable of displaying images. The display unit 331 constitutes the above-described face portion 305 of the guide robot 3, and the display unit 331 displays the above-described pair of pseudo eyes 306, 306, character images, and the like. The display unit 331 may be configured to be able to display the pair of pseudo eyes 306, 306, character images, and the like, and is configured by, for example, a liquid crystal panel, a projector, a screen, and the like.

The camera unit 34 is a camera having an imaging element such as a CCD or CMOS, and is configured to be able to capture an image of a visitor visiting the showroom 1 . The camera unit 34 is provided on the head 301 of the guide robot 3, for example. By providing the camera unit 34 on the head 301, it becomes easier to capture an image of the visitor's face. In addition, the camera unit 34 is preferably provided near the pair of pseudo eyes 306, 306 of the guide robot 3 from the viewpoint of capturing an image of the visitor's face.

The traveling unit 35 is composed of the above-described traveling device 304 that allows the guide robot 3 to travel by itself. The running unit 35 has a battery and a motor, and is configured to run by driving the motor with electricity from the battery. The running portion 35 can be configured using known electric technology. The sensor unit 36 detects the running and stopped states of the guide robot 3, such as a running speed sensor, an acceleration sensor, and a gyro sensor; It has various sensors such as a sensor for detection.

The storage unit 37 has a volatile or non-volatile memory (not shown). The storage unit 37 stores various programs executed by the calculation unit 38, various data, and the like. In addition, the storage unit 37 temporarily stores data and the like regarding the content of customer service with the visitor. For example, the visitor's requirements asked by the guide robot 3, the explanation given by the guide robot 3 to the visitor, and the like are temporarily stored.

The storage unit 37 stores a showroom database 371 and a communication database 372 as an example of a functional configuration of memory. The showroom database 371 stores, for example, data corresponding to the arrangement of the display vehicles 2 and tables arranged in the showroom 1 . The showroom database 371 is referred to when the guide robot 3 moves within the showroom. The communication database 372 stores data and the like corresponding to voice recognition processing and voice output processing when the guide robot 3 communicates with a visitor. The communication database 372 is referred to when the guide robot 3 communicates with the visitor.

The calculation unit 38 has a CPU, and stores signals received from the outside of the guide robot 3 via the communication unit 31, signals input via the input unit 32, signals detected by the sensor unit 36, and stored in the storage unit 37. It executes predetermined processing based on various programs, various data, etc., and outputs predetermined control signals to the communication section 31 , the output section 33 , the camera section 34 , the running section 35 and the storage section 37 .

For example, the calculation unit 38 outputs control signals to the traveling unit 35 and the storage unit 37 based on signals received from the server device 4 via the communication unit 31 and signals detected by the sensor unit 36 . By this processing in the calculation unit 38, the guide robot 3 is dispatched to the visitor. Further, for example, the calculation unit 38 outputs control signals to the camera unit 34 and the communication unit 31 based on signals received from the server device 4 via the communication unit 31 . By this processing in the calculation unit 38 , the visitor's face is imaged, and the imaged face image is transmitted to the server device 4 .

Further, for example, the calculation unit 38 outputs a control signal to the output unit 33 (display unit 331) based on the signal received from the server device 4 via the communication unit 111. FIG. By this processing in the calculation unit 38, the facial expression of the guide robot 3 is changed, and the line of sight of the pair of pseudo eyes 306, 306 is changed. Further, for example, the calculation unit 38 outputs control signals to the output unit 33 and the storage unit 37 based on the signal input via the input unit 32 . This processing by the calculation unit 38 enables the guide robot 3 to communicate with the visitor.

As shown in FIG. 4 , the server device 4 has a communication section 41 , an input section 42 , an output section 43 , a storage section 44 and a calculation section 45 . The server device 4 can be configured using a virtual server function on the cloud, or can be configured by distributing each function.

The communication unit 41 is configured to be able to wirelessly communicate with the imaging unit 11 and the guide robot 3 via the communication network 5 (see FIG. 3). The input unit 42 has various switches such as a touch panel and a keyboard that can be operated by the user, a microphone that can input voice, and the like. In addition, a user here is a salesclerk of a car dealer in this embodiment. The output unit 43 has, for example, a monitor capable of displaying characters and images, a speaker capable of outputting sound, and the like.

The storage unit 44 has a volatile or non-volatile memory (not shown). The storage unit 44 stores various programs executed by the calculation unit 45, various data, and the like. The storage unit 44 has a guide robot database 441, a showroom database 442, and a visitor database 443 as functional components of memory.

The guide robot database 441 stores basic information about the guide robot 3 such as the robot ID of the guide robot 3 used for the robot dispatch guide service, maintenance information, and the like. The showroom database 442 stores data corresponding to the arrangement of the display vehicles 2 and tables arranged in the showroom 1 . The showroom database 442 has the same configuration as the showroom database 371 stored in the storage unit 37 of the guide robot 3, and may have either configuration.

The visitor database 443 stores visitor information of visitors who visit the showroom 1 . The visitor information includes the visitor's basic information such as the visitor's address, name, age, occupation, and sex, as well as the visitor's face image and visit history. The visit history includes not only the content of business negotiations at the time of the visit, but also chat before the business negotiations.

In addition, the visitor database 443 stores the visitor in association with the visitor group when the visitor constitutes a group and visits. For example, when a visitor comes as a family group, the visitor is associated with the family group and stored, and when the visitor comes as a friend group, the visitor is associated with the friend group. memorize. A visitor may belong to a plurality of groups in this way, and a visitor map is formed in the visitor database 443 .

The computing unit 45 has a CPU and processes signals received via the input unit 42, signals received from outside the server device 4 via the communication unit 41, and various programs and data stored in the storage unit 44. Based on this, a predetermined process is executed, and a control signal is output to the communication section 41 , the output section 43 and the storage section 44 .

As shown in FIG. 4 , the processor 45 includes an in-facility image acquisition unit 451 , a robot image acquisition unit 452 , a robot line-of-sight instruction unit 453 , a visitor identification unit 454 , a visitor identification unit 454 , and a visitor identification unit 454 . It has a person group identification unit 455 , a group emotion identification unit 456 and a robot movement instruction unit 457 .

The in-facility image acquisition unit 451 acquires in-showroom images captured by the plurality of imaging units 11 installed in the showroom 1 . Specifically, the in-facility image acquisition unit 451 acquires image data (including still images and moving images) of the interior of the showroom 1 (the space where the exhibition vehicle 2 is displayed) captured by the plurality of imaging units 11, Input via the communication unit 41 . In the present embodiment, the in-facility image acquisition unit 451 causes the plurality of imaging units 11 to capture images of the inside of the showroom, and acquires the captured images of the inside of the showroom. Specifically, the in-facility image acquisition unit 451 outputs a control signal for causing the plurality of image capturing units 11 to image the showroom 1 via the communication unit 41, and communicates the showroom image data captured by the plurality of image capturing units 11. Input via the unit 41 .

The robot image acquisition unit 452 acquires an image including a visitor's face image captured by the guide robot 3 arranged in the showroom 1 . Specifically, the robot image acquisition unit 452 inputs image (including still images and moving images) data including the face image of the visitor captured by the guide robot 3 via the communication unit 41 . In this embodiment, the robot image acquisition unit 452 causes the guide robot 3 to image the visitor's face, and acquires an image including the imaged face image. Specifically, the robot image acquisition unit 452 outputs a control signal for causing the guide robot 3 to image the visitor's face via the communication unit 41, and the image data including the visitor's face image captured by the guide robot 3 is generated. is input through the communication unit 41 .

The robot line-of-sight instruction unit 453 indicates the line-of-sight direction of the pair of pseudo eyes 306 and 306 of the guide robot 3 . Specifically, the robot line-of-sight instruction unit 453 outputs a control signal for instructing the positions and movements of the pair of pseudo eyes 306 and 306 of the guide robot 3 to the guide robot 3 via the communication unit 41 .

When the control signal is input via the communication unit 31, the guide robot 3 controls the display unit 331 based on the input control signal to change the positions of the pair of pseudo eyes 306,306. In other words, move your line of sight. When the guide robot 3 moves its line of sight, the visitor is attracted by the line of sight of the guide robot 3 and looks in the direction of the line of sight of the guide robot 3. - 特許庁In this way, by moving the line of sight of the guide robot 3, it is possible to guide the line of sight of the visitor and encourage a change in the position or posture of the visitor. For example, by directing the line of sight of the guide robot 3 to the imaging unit 11 , the visitor's line of sight can be directed to the imaging unit 11 . Then, when the visitor looks at the imaging unit 11, the imaging unit 11 can capture an image of the visitor's face.

The visitor identification unit 454 identifies a visitor who has visited the showroom 1 from the in-showroom image acquired by the in-facility image acquisition unit 451 . For example, the visitor identification unit 454 extracts a person from the in-showroom image, and further extracts (recognizes) a face image from the extracted person. The visitor data stored in the visitor database 443 is searched for visitor data having a face image that matches the extracted face image, and the visitor is specified. If there is no visitor data having a face image that matches the extracted face image, the visitor is stored in the visitor database 443 as a new visitor.

When the visitor identification unit 454 determines that the visitor cannot be identified because the face image cannot be extracted (recognized) from the person extracted from the in-showroom image, the visitor identification unit 454 outputs a control signal to the robot movement instruction unit 457 . When receiving this control signal, the robot movement instructing section 457 instructs the guide robot 3 to move so that the guide robot 3 is dispatched under this person. Then, the robot image acquiring unit 452 causes the guide robot 3 to take an image of the person's face. The image including the face image of the person captured by the guide robot 3 is input to the robot image acquisition unit 452 via the communication unit 41, and the visitor identification unit 454 identifies the person input to the robot image acquisition unit 452. The facial image is used to identify the visitor in a manner similar to that described above.

The visitor group identification unit 455 identifies a visitor group who has visited the showroom 1 from the in-showroom image acquired by the in-facility image acquisition unit 451 . For example, the visitor group identification unit 455 extracts a plurality of visitors from the in-showroom image, and determines whether or not these visitors form a group based on the sense of distance or face orientation of these visitors. For example, a plurality of visitors who keep a certain distance for a predetermined time or longer are specified as a visitor group. Also, for example, a plurality of visitors who face each other in a conversation direction are identified as a visitor group. At this time, the visitor group identifying unit 455 also identifies the number of visitors who constitute the visitor group.

The visitor group identification unit 455 also identifies whether or not the plurality of visitors identified by the visitor identification unit 454 form a visitor group. For example, the visitor group identification unit 455 extracts visitor group information from the visitor information stored in the visitor database 443 of each visitor identified by the visitor identification unit 454, and extracts the visitor group information from the extracted visitor group information. Specifies whether the visitor constitutes a visitor group.

The group emotion specifying unit 456 estimates the emotions of a plurality of visitors forming the visitor group specified by the visitor group specifying unit 455 based on the in-showroom image, and specifies the group emotion of the visitor group. The group emotion identification unit 456 identifies, for example, whether the visitor group is in a good mood (for example, a happy group) or a bad mood group (for example, a frustrated or angry group). .

The group emotion identification unit 456 identifies the group emotion based on the estimated ratio of emotions of the plurality of visitors. For example, the group emotion identification unit 456 identifies a group in which the percentage of visitors in a good mood is high (the percentage of visitors in a bad mood is low) as a group in a good mood, and identifies a group in which the percentage of visitors in a bad mood is high. Identify the group (low percentage of happy visitors) as the bad group. That is, the ratio of emotions in this embodiment means the ratio of each emotion (bad mood, good mood, etc.) in the group.

For example, as shown in FIG. 1, the group emotion identification unit 456 has a friend group 101 made up of three visitors A, B, and C, and a parent-child group 102 made up of two visitors D, E. When the showroom 1 is visited, the group emotions of the friend group 101 and the parent-child group 102 are identified from the images of the showroom captured by the plurality of imaging units 11 . Group emotion identification unit 456 estimates the emotions of visitors A to C that make up friend group 101. For example, when it is estimated that visitors A and B are in a good mood and visitor C is in a bad mood, The group feeling of the friend group 101 is specified as "slightly good mood". Similarly, the group emotion identification unit 456 estimates the emotions of the visitors D and E that make up the parent-child group 102, and If it is estimated that the person is in a bad mood, the group emotion of the parent-child group 102 is specified as "slightly in a bad mood."

For group emotions, for example, only visitors in a good mood are in a good mood, visitors in a good mood are relatively in a good mood, and visitors in a good mood and visitors in a bad mood are equally likely to be in a good mood. "Normal", "Slightly bad mood" with relatively many visitors in a bad mood, and "Bad mood" only with visitors in a bad mood. At this time, if the visitor is a child, the group emotion may be specified so as to emphasize the emotion of the child visitor. may be specified.

The emotion of each visitor can be estimated based on the visitor's gestures, behavior, and the like. For example, the emotion of the visitor is estimated from the face image of the visitor extracted from the image inside the showroom, the behavior of the visitor, and the like. For example, if the visitor looks angry, looks around, or is unsettled by extortion, etc., it is assumed that the visitor is in a bad mood. On the other hand, if the visitor looks happy or is having a conversation with another visitor, it is estimated that the visitor is in a good mood.

Also, the group emotion specifying unit 456 estimates the visitor's emotion based on the robot-captured image acquired by the robot image acquisition unit 452, and updates the group emotion of the visitor group. For example, when the guide robot 3 is dispatched to the visitor group and the visitor is imaged by the guide robot 3 because the visitor cannot be specified based on the showroom image, the group emotion identification unit 456 3, the emotion of the visitor is estimated based on the face image of the visitor captured by the guide robot 3, and the group emotion is updated based on this.

The robot movement instruction unit 457 instructs the guide robot 3 to move so that the guide robot 3 is dispatched under the visitor group based on the group emotion specified by the group emotion specification unit 456 . Specifically, the robot movement instruction unit 457 outputs a control signal to the guide robot 3 via the communication unit 41 to instruct the guide robot 3 to move under the visitor group.

At this time, the robot movement instruction unit 457 gives priority to the visitor group identified by the group emotion identification unit 456 and whose group emotion is relatively bad, and instructs the guide robot to move below the visitor group. For example, if there are a visitor group whose group feeling is "bad mood" and a visitor group whose group feeling is "slightly bad mood", the visitor group whose group emotion is "bad mood" is prioritized. The guide robot 3 is instructed to move under the visitor group.

Further, when there are a plurality of visitor groups having the same group emotion specified by the group emotion specifying unit 456, the robot movement instruction unit 457 gives priority to a visitor group with a small number of people, and places the visitor group under the visitor group. to instruct the movement of the guide robot 3. For example, if there are two visitor groups whose group feeling is "slightly bad mood", one visitor group consists of three people, and the other visitor group consists of five people. gives priority to a visitor group consisting of three people, and instructs the movement of the guide robot 3 under the visitor group.

Furthermore, when the visitor specifying unit 454 determines that the visitor cannot be specified based on the in-showroom image, the robot movement instructing unit 457 dispatches the guide robot 3 to the visitor who is determined to be unspecifiable. The guide robot 3 is instructed to move. For example, when the robot movement instruction unit 457 receives a control signal output from the visitor identification unit 454 indicating that the visitor cannot be identified, the guide robot 3 moves near the person extracted from the showroom image by the visitor identification unit 454. The guide robot 3 is instructed to move so as to be dispatched.

At this time, if the person extracted from the in-showroom image by the visitor identification unit 454 constitutes a visitor group, the robot movement instruction unit 457 dispatches the guide robot 3 to the vicinity of the visitor group to which the person belongs. The guide robot 3 is instructed to move. Then, the robot movement instruction unit 457 performs an operation to prompt a change in the position or posture of the visitor determined to be unidentifiable so that the visitor can be identified by the in-showroom image acquired by the in-facility image acquisition unit 451 . The guide robot 3 is instructed. For example, the robot movement instruction unit 457 instructs the guide robot 3 to move so that the visitor determined to be unidentifiable faces the imaging unit 11 in a position or posture.

For example, the robot movement instruction unit 457 instructs the guide robot 3 to move under the visitor determined to be unidentifiable, with the imaging unit 11 positioned behind the guide robot 3 . A visitor determined to be unidentifiable looks at the guide robot 3 when he/she notices the approaching guide robot 3, so that the visitor's face can be imaged by the imaging unit 11 positioned behind the guide robot 3. - 特許庁At this time, it is preferable to make the guide robot 3 perform an action that is easy for the visitor determined to be unidentifiable to notice. In this way, a change in the position or posture of the visitor is encouraged.

Further, the robot line-of-sight instruction unit 453 instructs the line-of-sight direction of the pair of pseudo eyes 306, 306 of the guide robot 3, thereby guiding the line of sight of the visitor and prompting a change in the position or posture of the visitor. is preferred. For example, it is preferable to guide the visitor's line of sight toward the imaging unit 11 by moving the direction of the line of sight of the pair of pseudo eyes 306 and 306 toward the imaging unit 11 . At this time, for example, the robot movement instructing unit 457 causes the guide robot 3 to perform a rotating motion or the like in conjunction with the movement of the line of sight, thereby making it easier to guide the line of sight.

FIG. 5 is a flow chart showing an example of the guide robot dispatching process executed by the computing unit 45 of the server device 4 of FIG. FIG. 6 is a flow chart showing an example of visitor identification processing executed by the calculation unit 45 of the server device 4 in FIG. 3, and FIG. 9 is a flowchart showing an example of group identification processing; The guide robot dispatching process shown in FIG. 5 is started, for example, when the showroom 1 is opened and is executed until the showroom is closed.

As shown in FIG. 5, first, in step S1, an in-facility image acquisition unit 451 acquires an in-showroom image captured by a plurality of imaging units 11 installed in the showroom 1 (acquisition of in-facility image). step). Next, in step S2, a visitor identification process is executed to identify a visitor who has visited the showroom 1 from the in-facility image acquired in the in-facility image acquisition step (visitor identification step).

In the visitor identification process (visitor identification step), as shown in FIG. Extract the person who visited. Next, in step S21, it is determined whether or not the face image of the extracted person can be extracted (recognized) by processing in the visitor identification unit 454. FIG. If the result in step S21 is NO, the process advances to step S22, where the robot movement instruction unit 457 instructs the guide robot 3 to move so that the guide robot 3 is dispatched under the person determined to be unextractable ( robot movement instruction step).

Next, in step S23, it is determined whether or not the facial image of the person determined to be unextractable can be acquired through the process of the visitor identification unit 454. FIG. If the result in step S23 is NO, the process proceeds to step S24, where the direction of the line of sight of the pair of pseudo eyes 306, 306 of the guide robot 3 is instructed by processing in the robot line of sight instruction unit 453 to guide the line of sight of the visitor. Prompting the visitor to change his/her position or posture (robot gaze direction step).

Next, in step S25, it is determined whether or not the facial image of the person determined to be unextractable can be obtained by the process of the visitor identification unit 454. FIG. If the result in step S25 is NO, the process proceeds to step S26, where the robot image acquisition unit 452 causes the guide robot 3 to take an image of the face and acquires the face image of the person determined to be unextractable (robot image acquisition step). .

On the other hand, if each of steps S21, S23, and S25 is affirmative, or if the face image of the person determined to be unextractable (identifiable) is acquired in step S26, then in step S27, the visitor identification unit The process at 454 extracts (recognizes) the face image of the visitor. Next, in step S28, the visitor is identified by processing in the visitor identification unit 454. FIG.

After executing the visitor identification process (visitor identification step), as shown in FIG. 5, next, in step S3, a visitor group identification process for identifying a visitor group is executed (visitor group identification step).

In the visitor group identification process (visitor group identification step), as shown in FIG. 7, first, in step S30, the visitor identified in the visitor identification step is identified by processing in the visitor group identification unit 455. do. Next, in step S31, it is determined whether or not the specified visitor constitutes a visitor group. If the result in step S31 is NO, the process returns to step S1. On the other hand, if the result in step S31 is affirmative, the process proceeds to step S32 to specify a visitor group.

Once the visitor group identification process is executed, as shown in FIG. 5, in step S4, the group emotion identification unit 456 estimates the emotions of each of the plurality of visitors (group emotion identification step). Next, in step S5, the group emotion identification unit 456 identifies the group emotion of the visitor group based on the estimated emotions of each of the plurality of visitors (group emotion identification step). Next, in step S6, the robot movement instructing unit 457 instructs the guide robot 3 to move so that the guide robot 3 is dispatched under the visitor group based on the group emotion of the visitor group ( robot movement instruction step).

Next, in step S7, it is determined whether or not the showroom 1 is closing time. On the other hand, if the result in step S7 is affirmative, the process ends.

According to this embodiment, the following effects can be obtained.
(1) A server device 4 that manages a self-propelled guide robot 3 that provides services to visitors visiting the showroom 1 of an automobile dealer. The server device 4 includes an in-facility image acquisition unit 451 that acquires an in-showroom image captured by a plurality of image pickup units 11 installed in the showroom 1, and an in-facility image acquisition unit 451 that acquires the in-facility image acquired by the in-facility image acquisition unit 451 to the showroom 1. It comprises a visitor identification unit 454 that identifies a visitor who has visited, and a robot movement instruction unit 457 that instructs the guide robot 3 to move so that the guide robot 3 is dispatched under the visitor. When the visitor specifying unit 454 determines that the visitor cannot be specified based on the in-showroom image, the robot movement instruction unit 457 instructs the robot movement instruction unit 457 to dispatch the guide robot 3 to the visitor who is determined to be unspecifiable. to instruct the guide robot 3 to move.

With this configuration, it is possible to appropriately and efficiently serve a visitor who visits the showroom 1 of the car dealer using the guide robot 3 . For example, by dispatching the guide robot 3 under the visitor who is determined to be unidentifiable, the visitor can be identified by the image in the showroom, prompting a change in the position or posture of the visitor, or guiding the visitor. The robot 3 can be made to take a visitor's face image.

For example, when the visitor turns to the guide robot 3 as the guide robot 3 moves, it becomes easier for the imaging unit 11 to capture an image of the visitor's face, and the guide robot 3 directly captures the visitor's face. be possible. This makes it possible to identify the visitor who has been determined to be unidentifiable. As a result, for example, it is possible to provide appropriate customer service by cooperating the imaging unit 11 and the guide robot 3 .

For example, it is possible to dispatch a guide robot to a visitor who is in a bad mood to communicate with the visitor. For example, by dispatching a guide robot to a visitor who has been kept waiting for a long time and is in a bad mood to communicate with the visitor, it is possible to make the visitor feel better or to prevent the visitor from being in a bad mood. As a result, subsequent customer service such as business negotiations by a salesperson (for example, a salesperson) of the automobile dealer can be smoothly performed.

Also, for example, it is possible to have the guide robot ask the visitor about their requirements in advance or provide simple guidance. For example, for a visitor who has been waiting for a long time or a group of visitors who are expected to wait for a long time, the guide robot can be made to ask requirements and the like or provide simple guidance in advance. This enables efficient customer service by a salesperson (for example, a salesperson) afterward.

Thus, by using the server device 4 according to the present embodiment, for example, efficient and smooth customer service using the guide robot 3 becomes possible.

(2) Further provided is a visitor group identification unit 455 that identifies a visitor group who has visited the showroom 1 from the in-facility image acquisition unit 451 acquired by the showroom image acquisition unit 451 . If the visitor group identified by the visitor group identifying unit 455 includes a visitor who is determined to be unidentifiable by the visitor identifying unit 454, the robot movement instruction unit 457 determines that the visitor group cannot be identified. The guide robot 3 is instructed to move so that the guide robot 3 is dispatched under the visitor group to which the received visitor belongs. For example, when a group visits the showroom 1, they often face each other and talk while visiting. In this case, it is difficult to recognize the visitor's face. It becomes possible to recognize the face of a visitor.

(3) The robot movement instruction unit 457 instructs the guide robot 3 to move to the vicinity of the visitor determined to be unidentifiable by the visitor identification unit 454, and the visitor determined to be unidentifiable obtains an image of the inside of the facility. An instruction is given to prompt a change in the position or posture of the visitor that can be identified by the image of the inside of the showroom acquired by the unit 451 . For example, the robot movement instruction unit 457 instructs the guide robot 3 to move so that the visitor determined to be unidentifiable faces the imaging unit 11 in a position or posture. As a result, it becomes possible to recognize the face of the visitor determined to be unidentifiable, and to identify the visitor determined to be unidentifiable.

(4) The guide robot 3 has a pair of pseudo eyes 306, 306 whose line of sight is movable, and further includes a robot line of sight instruction unit 453 that indicates the direction of the line of sight of the pair of pseudo eyes 306, 306 of the guide robot 3. The robot line-of-sight instruction unit 453 directs the line-of-sight direction of the pair of pseudo eyes 306, 306 of the guide robot 3 moved to the vicinity of the visitor by the robot movement instruction unit 457, thereby guiding the visitor's line of sight. Prompt a change in a person's position or posture. This makes it easier to recognize the face of the visitor who is determined to be unidentifiable, and allows the visitor who is determined to be unidentifiable to be identified.

(5) A robot management method for managing a self-propelled guide robot 3 that provides services to visitors visiting the showroom 1 of an automobile dealer. The robot management method includes an in-facility image acquisition step in which the server device 4 acquires an in-facility image captured by a plurality of imaging units 11 installed in the showroom 1, and an in-facility image acquisition step in which the in-facility image is acquired. A visitor identification step for identifying a visitor who has visited the showroom from the display device, and a robot movement instruction step for instructing the movement of the guidance robot 3 so that the guidance robot 3 is dispatched under the visitor are executed. In the robot movement instruction step, when it is determined that the visitor cannot be specified based on the image in the showroom in the visitor specification step, the guide robot 3 is dispatched to the visitor who is determined to be unspecifiable. to instruct the guide robot 3 to move.

By this method, like the server device 4 described above, it is possible to appropriately and efficiently serve visitors who visit the showroom 1 of the car dealer using the guide robot 3 . For example, efficient and smooth customer service using the guide robot 3 is possible.

(6) The robot management system 100 includes the server device 4 described above, a plurality of imaging units 11 installed in the showroom 1 and capable of communicating with the server device 4, and the server device 4 arranged in the showroom 1 and capable of self-running. and a guide robot 3 capable of communication. As a result, efficient customer service using the guide robot 3 can be appropriately performed for visitors visiting the showroom 1 of the automobile dealer. For example, efficient and smooth customer service using the guide robot 3 is possible.

In the above-described embodiment, the computing unit 45 of the server device 4 includes the in-facility image acquisition unit 451, the robot image acquisition unit 452, the robot line-of-sight instruction unit 453, the visitor identification unit 454, and the visitor group identification unit 455. , the group emotion specifying unit 456 and the robot movement instructing unit 457, but the present invention is not limited to this. The calculation unit 45 may include an in-facility image acquisition unit 451 , a visitor identification unit 454 , and a robot movement instruction unit 457 . The calculation unit 45 preferably includes a visitor group identification unit 455 , and more preferably includes a robot line-of-sight instruction unit 453 .

In the above-described embodiment, the guidance robot dispatching process executed by the calculation unit 45 of the server device 4 includes an in-facility image acquisition step, a visitor identification step, a visitor group identification step, a group emotion identification step, and a robot movement. However, the present invention is not limited to this. The guide robot dispatching process may include an in-facility image acquisition step, a visitor identification step, and a robot movement instruction step in the visitor identification step.

The above description is merely an example, and the present invention is not limited by the above-described embodiments as long as the features of the present invention are not impaired.

1 showroom (facility), 3 guide robot, 4 server device (robot management device), 5 communication network, 11 imaging unit, 100 robot management system, 101 friend group (visitor group), 102 parent-child group (visitor group), 306 pair of pseudo eyes, 451 in-facility image acquisition unit, 452 robot image acquisition unit, 453 robot gaze instruction unit, 454 visitor identification unit, 455 visitor group identification unit, 456 group emotion identification unit, 457 robot movement instruction unit, A to E Visitors

Claims (5)

A robot management device that manages a self-propelled guide robot that provides services to visitors visiting a facility,
an in-facility image acquisition unit that acquires an in-facility image captured by an imaging unit installed in the facility;
a visitor identification unit that identifies a visitor who has visited the facility from the in-facility image acquired by the in-facility image acquisition unit;
a visitor group identifying unit that identifies a group of visitors who have visited the facility from the in-facility image acquired by the in-facility image acquiring unit;
a robot movement instruction unit that instructs the guide robot to move so that the guide robot is dispatched under the visitor;
When the visitor identification unit determines that the visitor cannot be identified based on the in-facility image, the robot movement instruction unit dispatches the guide robot to the visitor who is determined to be unidentifiable. When the visitor group identified by the visitor group identification unit includes a visitor who is determined to be unidentifiable by the visitor identification unit, 1. A robot management apparatus, comprising: a robot management apparatus that instructs a guide robot to move so that the guide robot is dispatched to a visitor group to which a visitor determined to be unidentifiable belongs. The robot management device according to claim 1,
The robot movement instruction unit instructs the guide robot to move to the vicinity of the visitor determined to be unidentifiable by the visitor identification unit, and the visitor determined to be unidentifiable is 1. A robot management apparatus, characterized in that it instructs an action that prompts a change in the position or posture of the visitor so as to be identifiable from an acquired in-facility image. In the robot management device according to claim 2,
The guide robot has a pseudo eye capable of moving a line of sight,
further comprising a robot line-of-sight instruction unit for instructing a line-of-sight direction of the pseudo eye of the guide robot;
The robot line-of-sight instruction unit instructs the direction of the line-of-sight of the pseudo eyes of the guide robot that has moved to the vicinity of the visitor by the robot movement instruction unit, thereby guiding the line of sight of the visitor. A robot management device characterized by prompting a change in position or posture. A robot management method for managing a self-propelled guide robot that provides services to visitors visiting a facility,
respectively executed by a computer,
an in-facility image acquisition step of acquiring an in-facility image captured by an imaging unit installed in the facility;
a visitor identification step of identifying a visitor who has visited the facility from the in-facility image acquired in the in-facility image acquisition step;
a visitor group identification step of identifying a visitor group who has visited the facility from the in-facility image acquired in the in-facility image acquisition step;
a robot movement instruction step of instructing the guide robot to move so that the guide robot is dispatched under the visitor;
In the robot movement instruction step, when the visitor identification step determines that the visitor cannot be identified based on the in-facility image, the guide robot is dispatched under the visitor who is determined to be unidentifiable. When the visitor group identified in the visitor group identification step includes a visitor determined to be unidentifiable by the visitor identification step 1. A robot management method comprising: instructing a guide robot to move so that the guide robot is dispatched under a visitor group to which a visitor determined to be unidentifiable belongs. a robot management device according to any one of claims 1 to 3;
an imaging unit installed in the facility and capable of communicating with the robot management device;
A robot management system comprising the guide robot disposed in the facility, capable of self-propelled communication, and capable of communicating with the robot management device.

Images