JP2019008158A - Decentralized system integrating and sharing information on the basis of ros architecture - Google Patents

Abstract

To provide a decentralized system for integrating and sharing information on the basis of ROS architecture.SOLUTION: According to the present invention, there is provided a decentralized system in which a plurality of modules integrate and share information on the basis of ROS (Robot Operating System) architecture, the system including an information management module which saves a message in response to a message storage request transmitted from each module, and then transmits, as a return value, a data identifier of the saved message to the module as the request source, wherein the information management module transmits, in response to a message acquisition request transmitted from each module and specifying a data identifier, a message regarding the data identifier to the module as the request source, and each module distributes, in response to receiving the data identifier of the saved message from the information management module, the data identifier to other modules.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a distributed system, and more particularly to a distributed system that integrates and shares information based on a ROS architecture.

  In recent years, in the field of artificial intelligence robots, in order to put it to practical use, it has a style and granularity such as information at the symbolic concept level (speech dialogue, emotion processing, knowledge inference) and physical signal level information (human / environmental sensing, motion). Various techniques for smoothly integrating and sharing different multimodal information have been studied.

  In this regard, Non-Patent Document 1 describes a comprehensive intelligence application that smoothly integrates and shares four types of elemental intelligence: knowledge inference, speech dialogue, image sensing of humans and objects, and operations based on the ROS (Robot Operating System) architecture. Disclose the development platform.

Takeshi Morita, Ryota Nishimura, Takahira Yamaguchi: Architecture of Integrated Intelligence Application Development Platform PRINTEPS Based on ROS, 30th Annual Conference of Japanese Society for Artificial Intelligence, 4C4-1 (2016)

  The present invention has been made in view of the above, and an object thereof is to provide a distributed system that integrates and shares information based on the ROS architecture.

  As a result of intensive studies on a distributed system that integrates and shares information based on the ROS architecture, the present inventor has conceived the following configuration and arrived at the present invention.

  That is, according to the present invention, a plurality of modules is a distributed system in which information is integrated and shared based on the ROS (Robot Operating System) architecture, and a message is transmitted in response to a message storage request transmitted from each module. An information management module for storing and transmitting the data identifier of the stored message to the requesting module as a return value, the information management module responding to a message acquisition request specifying the data identifier transmitted from each module In response to receiving the data identifier of the stored message from the information management module, each module transmits the message relating to the data identifier to the requesting module. Distributed system It is.

  As described above, according to the present invention, a distributed system that integrates and shares information based on the ROS architecture is provided.

The figure which shows the functional block of the distributed system of this embodiment. The system block diagram of "robot cafe". The sequence diagram of the process which the distributed system of this embodiment performs. The figure which shows the data structure of the message which each module transmits. The figure for demonstrating the cooperation aspect between modules. The figure which shows the functional block of the voice interaction module of this embodiment.

  Hereinafter, the present invention will be described with reference to embodiments shown in the drawings, but the present invention is not limited to the embodiments shown in the drawings. In the drawings referred to below, the same reference numerals are used for common elements, and the description thereof is omitted as appropriate.

  FIG. 1 conceptually shows functional blocks of a distributed system 100 according to an embodiment of the present invention. As shown in FIG. 1, the distributed system 100 according to the present embodiment includes an information management module 10, an image processing module 20, a voice interaction module 30, a knowledge processing module 40, and a robot control module 50. Each software module is configured to integrate and share information based on a ROS (Robot Operating System) architecture. The module configuration shown in FIG. 1 is merely an example, and the distributed system 100 according to the present embodiment includes at least an information management module 10 and two or more other software modules (hereinafter simply referred to as modules). It only has to be included.

  Here, before entering into a detailed description of the distributed system 100 of the present embodiment, data communication based on the ROS architecture will be outlined.

  ROS (Robot Operating System) is a robot framework that provides libraries and tools to assist software developers in creating robot applications and is being developed and maintained by the Open Source Robotics Foundation. ROS provides a meta-operating system that enables inter-process communication of execution programs executed on different OSs (Windows, Mac, Linux, Android, iOS, etc.) and different hardware. According to ROS, it becomes easy to integrate a plurality of software modules that operate on different OSs and hardware, and a distributed system such as a multi-robot and a multi-sensor can be easily constructed.

  In ROS, each execution program executed on a distributed system is the minimum component called a “node”, and data called a “message” is exchanged between the nodes. “Message” is data independent of programming language defined in Message type. Message type can have multiple variables and arrays like structures, and each variable type is prepared in advance. In addition to using types (such as int8 and string), you can also define your own.

  In ROS, two types of communication methods, topic and service, are prepared as message communication methods. These two communication methods are used according to the purpose, message format, and time granularity. Can be used properly.

  A topic is an asynchronous communication method. In ROS, sending a message defined by a topic is called “publish”, and a node to be delivered is called a “publisher”. In addition, receiving a message defined by a topic is called a “subscribe”, and a node that subscribes is called a “subscriber”. In ROS, this topic can be used to continuously transmit desired data from a publisher to a subscriber.

  On the other hand, the service is a synchronous communication method. In ROS, a node that requests a message defined by a service is called a service client, and a node that responds to this request is called a service server. In ROS, a service client and a service server are connected only once as necessary, and the connection is disconnected each time a request / response is executed.

  In ROS, a name server called “master” is prepared to manage connections between nodes. Each node registers information such as its own subscriber node name, topic name to be subscribed, its own distributor node name, topic name to be delivered, service name, message type name, URI address and port in the master. To do. On the other hand, each node acquires information on other nodes from the master, and directly exchanges messages with other nodes based on the acquired information.

  As described above, the data communication based on the ROS architecture has been outlined, and then the details of specific processing executed by the distributed system 100 of the present embodiment will be described.

  Hereinafter, a case where the distributed system 100 of this embodiment is applied to a “robot cafe” will be described as an example.

  FIG. 2 shows a system configuration diagram of the “robot cafe”. As shown in FIG. 2, in the “robot cafe”, a photographing device 70a is installed to photograph the state of the entrance in the store, and photographing devices 70b and 70c are disposed to photograph the state of the table. Microphones 80a and 80b are installed on each table to record audio. A customer service robot 60a and a serving robot 60b are arranged on the floor, and a cooking robot 60c is arranged in the kitchen. Here, the built-in speaker 90 is mounted on the customer service robot 60a.

  In the “robot cafe”, the five modules shown in FIG. 1 are distributed on two or more computers connected by a network, and the five modules cooperate with each other through the integration and sharing of information ( The following six types of services are realized: 1) greetings when entering a store, (2) seat guidance, (3) ordering, (4) preparation and transportation of food and drink, (5) accounting, and (6) postponing. Here, the role that the five modules shown in FIG. 1 play in the “robot cafe” will be described.

  The knowledge processing module 40 performs various knowledge inferences based on information acquired by the image processing module 20 and the voice interaction module 30 using workflow, business rules, ontology, and the like.

  The image processing module 20 detects the customer's entry based on the captured images acquired by the imaging devices 70a, 70b, and 70c, and acquires the number, age, sex, attributes, and the like of the customer.

  The voice interaction module 30 understands the utterance contents of the customer from the voices acquired by the microphones 80a and 80b, and outputs appropriate response contents corresponding thereto as synthesized voices to the built-in speaker 90 of the customer service robot 60a.

  The robot control module 50 controls the movement of the customer service robot 60a in the store, the food distribution operation of the food distribution robot 60b, and the food preparation operation by the cooking robot 60c.

  On the other hand, the information management module 10 stores messages transmitted from the above-described modules in a database, and reads and provides messages requested by each module from the database. The database described above is configured as a document-oriented database, and messages are stored in the document-oriented database in the form of JSON (JavaScript Object Notation), and are read and written in the form of Message type data. .

  The role of the five modules has been described above. Next, the contents of processing executed by each module for information integration / sharing will be described based on the sequence diagram shown in FIG.

  In the following, a case where a message having the service name “customer information” is integrated and shared between modules will be described. Here, as shown in FIG. 4A, “customer information” includes (1) data ID, (2) number of people, (3) configuration, (4) attribute, (5) preference, (6) allergy, (7) Seven variables are recommended.

  In the following description, it is assumed that all modules illustrated in FIG. 1 are registered in the master as a publisher and a subscriber for a message having a topic name “data ID”.

  The image processing module 20 acquires the number of customers and the configuration (age, gender) based on the captured image acquired from the imaging device 70 (S1).

  Next, the image processing module 20 stores the information acquired in S1 as the value of the “customer information” variable (S2). FIG. 4B shows a message in which the value “1 person” is stored in the variable “number of people” and the value “male in 30s” is stored in the variable “configuration”.

  Next, the image processing module 20 transmits a message storage request for the message shown in FIG. 4B to the information management module 10 as a service (S3).

  In response, the information management module 10 issues a data identifier (hereinafter referred to as data ID) to the received message (S4), and stores the issued data ID in the variable “data ID” of the message. . FIG. 4C shows a message in which the value “12345” is stored in the variable “data ID”.

  Thereafter, the information management module 10 saves the message storing the data ID in the document-oriented database (S5), and transmits the data ID “12345” issued in S4 to the requesting image processing module 20 as a return value (S5). S6).

  In response to this, the image processing module 20 stores the data ID “12345” received from the information management module 10 as the value of the variable “data ID” of the message having the topic name “data ID” (S7). FIG. 4E shows a message in which the value “12345” is stored in the variable “data ID”.

  Next, the image processing module 20 which is a publisher of the message having the topic name “data ID” sends the message shown in FIG. 4E to the knowledge processing module 40 registered as the subscriber. Distribute by topic (S8).

  In response to this, the knowledge processing module 40 transmits a message acquisition request specifying the value “12345” of the variable “data ID” of the received message to the information management module 10 as a service (S9).

  In response to this, the information management module 10 searches the document-oriented database using the value “12345” of the variable “data ID” of the received message as a key, and extracts the message shown in FIG. 4C (S10). Thereafter, the information management module 10 transmits the extracted message to the requesting knowledge processing module 40 (S11).

  In response to this, the knowledge processing module 40 performs knowledge inference based on the values of the variables “number of people” and “configuration” of the received message (see FIG. 4C) and estimates the “attribute” of the customer ( S12) The estimated result “salaryman” is stored as the value of the variable “attribute” of the received message (S13). FIG. 4D shows a message in which the value “company employee” is stored in the variable “attribute”.

  Next, the knowledge processing module 40 transmits a message storage request for the message shown in FIG. 4D to the information management module 10 as a service (S14). In response to this, the information management module 10 saves the received message in the document-oriented database (S15), and transmits the data ID “12345” of the saved message to the knowledge processing module 40 of the request source as a return value (S15). S16).

  In response, the knowledge processing module 40 stores the data ID “12345” received from the information management module 10 as the value of the variable “data ID” of the message having the topic name “data ID” (S17). FIG. 4E shows a message in which the value “12345” is stored in the variable “data ID”.

  Next, the knowledge processing module 40, which is the publisher of the message with the topic name “data ID”, sends the message shown in FIG. 4 (e) to the voice interaction module 30 registered as the subscriber. Distribute by topic (S18).

  As described above, in the present embodiment, each module stores the information acquired by itself in the value of the “customer information” variable, and stores the “customer information” in the information management module 10. The data ID of the stored “customer information” is distributed to another module (subscriber module). On the other hand, the subscriber module to which the data ID has been distributed acquires “customer information” related to the distributed data ID from the information management module 10, and uses the acquired information as the variable value of the acquired “customer information”. The stored “customer information” is stored in the information management module 10, and the data ID of the stored “customer information” is distributed to other modules (subscriber modules). In this embodiment, the series of processes described above are repeatedly executed, so that the information acquired by each module is integrated in “customer information” and shared between modules.

  The processing executed by each module for information integration / sharing has been described above. Next, the cooperation mode between modules will be described based on a specific example shown in FIG.

  The image processing module 20 writes the number of customers (2 persons) and the configuration (female in the 30s + infant male) acquired based on the captured image acquired from the imaging device 70 in the “customer information” and stores them in the information management module 10. At the same time, the data ID “23456” of the stored “customer information” is distributed to the knowledge processing module 40.

  The knowledge processing module 40 acquires “customer information” related to the data ID (23456) distributed from the image processing module 20 from the information management module 10, and acquires the values of the acquired “customer information” variables (number of people: 2 people, Composition: Estimate customer attributes (with children) by knowledge reasoning based on 30s (female + infant male). Thereafter, the knowledge processing module 40 stores the estimated attribute (with children) in the value of the variable “attribute” of “customer information” and saves it in the information management module 10, and at the same time, the data ID “ 23456 "is distributed to the voice interaction module 30.

  The voice interaction module 30 acquires “customer information” related to the data ID (23456) distributed from the knowledge processing module 40 from the information management module 10, and sets “child” as the value of the variable “attribute” of the acquired “customer information”. In response to the storage of “accompanied”, the user interacts with the customer based on a predetermined dialogue rule to acquire allergy information of the child. Thereafter, the voice interaction module 30 stores the acquired allergy information “wheat” in the value of the variable “allergy” of “customer information” and saves it in the information management module 10, and at the same time, the data ID “ 23456 "is distributed to the knowledge processing module 40.

  The knowledge processing module 40 acquires “customer information” related to the data ID (23456) distributed from the voice interaction module 30 from the information management module 10 and the value “wheat” of the variable “allergy” of the acquired “customer information”. Based on knowledge reasoning, “rice flour cake” is estimated as a recommended product for customers. Thereafter, the knowledge processing module 40 stores the estimated recommended product “rice flour cake” in the value of the “customer information” variable “recommended” and stores it in the information management module 10, and at the same time, stores the stored “customer information”. The data ID “23456” is distributed to the voice interaction module 30.

  The voice interaction module 30 acquires “customer information” related to the data ID (23456) distributed from the knowledge processing module 40 from the information management module 10, and sets “0” as the value of the variable “recommended” of the acquired “customer information”. In response to the storage of “rice flour cake”, a synthesized speech “How about rice flour cake?” Is generated based on a predetermined matching rule. Thereafter, the voice interaction module 30 outputs the generated synthesized voice to the built-in speaker 90 of the customer service robot 60a.

  As described above, according to the present embodiment, a module that does not have a technique for directly interacting with the user cooperates with the voice interaction module 30 via the information management module 10 to reflect the explicit intention of the user. Appropriate actions can be taken.

  As described above, according to the present embodiment, information that cannot be acquired independently by each module is accumulated in the information management module 10 while complementing information between the modules, thereby integrating and sharing information in the distributed system 100. Is done.

  In the present embodiment, in addition to this, the following cooperation is assumed. For example, when the image processing module 20 acquires the number of customers based on the captured image, the result is prevented from shaking for each frame by cooperation with the knowledge processing module 40 (use of stream inference), and stable number detection is possible. Is possible. Further, by linking the user's behavior estimation (estimation during eating / speaking / speaking) based on the photographed image by the image processing module 20 and the voice interaction module 30, it is possible to estimate the speaker more robustly. On the other hand, by using the customer information acquired by the image processing module 20, the voice dialogue module 30 can greatly simplify the contents of dialogue with a user who tends to be redundant for information confirmation. A more natural dialogue will be made.

  As described above, the cooperation of the modules shown in FIG. 1 has been described. Next, the cooperation of the submodules constituting the voice interaction module 30 will be described.

  As shown in FIG. 6, the speech dialogue module 30 of this embodiment includes a speech acquisition module 31, a speech recognition module 32, a language understanding module 33, a dialogue management module 34, a response generation module 35, and a speech synthesis module. It consists of six submodules of 36.

  The voice acquisition module 31 passes the voice waveform data acquired by the microphones 80 a and 80 b to the voice recognition module 32. In response to this, the speech recognition module 32 sequentially executes speech recognition, and passes a natural language sentence (text) as a result to the language understanding module 33. In response to this, the language understanding module 33 converts the received text into an internal representation for understanding by the system based on a predetermined matching rule, and passes it to the dialogue management module 34.

  In response to this, the dialogue management module 34 converts the received internal representation into an internal representation of the system response and passes it to the response generation module 35. In response to this, the response generation module 35 converts the received internal expression into a natural language sentence (text) based on a predetermined matching rule, and passes it to the speech synthesis module 36. In response to this, the speech synthesis module 36 synthesizes the received natural language sentence (text) speech and outputs the synthesized speech to the built-in speaker 90 of the customer service robot 60a.

  Here, in this embodiment, a part or all of the submodules shown in FIG. 6 are configured to store their latest internal state as a message in the information management module 10, and each submodule is The internal state stored by other submodules and the internal state stored by itself can be referred to via the information management module 10.

As a result, for example, the dialog management module 34 receives the latest system response (for example, [ConfToUser: 3]) stored in the information management module 10 during the dialog with the user, and the language understanding module 33 thereafter. By combining the latest user input (for example, [Num: 4]) and performing interactive processing based on the state transition rule, a response (for example, [ConfToUser: 4]) in consideration of the context can be generated. It becomes like this. As a result, a natural dialogue that is not a question-and-answer type as exemplified below is realized.

System: “Come on. Three people?”
User: “No, 4 people.”
System: “Are you four people?”
User: “Yes.”
System: “We will guide you. Please wait a moment.”

  As described above, the present invention has been described with the embodiment. However, the present invention is not limited to the above-described embodiment, and the functions and effects of the present invention are within the scope of other embodiments that can be considered by those skilled in the art. As long as it plays, it is included in the scope of the present invention.

  Note that each function of the above-described embodiment can be realized by a device-executable program described in C, C ++, C #, Java (registered trademark), and the like. The program of this embodiment includes a hard disk device, a CD-ROM. , MO, DVD, flexible disk, EEPROM, EPROM and the like can be stored and distributed in a device-readable recording medium, and can be transmitted via a network in a format that other devices can.

DESCRIPTION OF SYMBOLS 10 ... Information management module, 20 ... Image processing module, 30 ... Voice dialogue module, 31 ... Voice acquisition module, 32 ... Speech recognition module, 33 ... Language understanding module, 34 ... Dialogue management module, 35 ... Response generation module, 36 ... Speech synthesis module, 40 ... Knowledge processing module, 50 ... Robot control module, 60 ... Robot, 70 ... Imaging device, 80 ... Microphone, 90 ... Built-in speaker, 100 ... Distributed system

Claims (5)

A distributed system in which a plurality of modules integrate and share information based on a ROS (Robot Operating System) architecture,
In response to a message storage request transmitted from each module, the information storage module stores the message and transmits the data identifier of the stored message as a return value to the requesting module;
The information management module includes:
In response to the message acquisition request specifying the data identifier transmitted from each module, the message related to the data identifier is transmitted to the requesting module,
Each module is
In response to receiving the data identifier of the stored message from the information management module, the data identifier is distributed to other modules.
Distributed system. Each said module is
Sending the message storage request and the message acquisition request to the information management module as a service,
Deliver the data identifier of the stored message to another module as a topic;
The distributed system according to claim 1. The information management module includes:
Storing a message according to the message storage request in a document-oriented database;
The distributed system according to claim 1 or 2. Each of the modules stores information acquired by itself in the value of a message variable related to the message storage request.
The distributed system as described in any one of Claims 1-3. The plurality of modules are:
Including a voice interaction module that at least understands a user's utterance based on voice input and outputs a response to the utterance as synthesized voice;
The distributed system as described in any one of Claims 1-4.