JP6944817B2 - Robot control system - Google Patents

Description

The present invention relates to a robot control system, and in particular, when a content consisting of a document or image data created in advance is reproduced to give a presentation or the like, the robot can deal with the content content by synchronizing with the content content to be reproduced. It relates to a robot control system capable of executing actions such as speech.

Conventionally, a presentation system has been used in which a person in charge creates contents including documents, drawings, images, etc. with an information processing device such as a personal computer, and displays and reproduces the contents in a set order. In addition, a digital signage system is also used in which content created on a user's mobile terminal is distributed to a signage device via a network, and the content is displayed and reproduced.

In addition, an information providing device has been proposed in which detailed data of the selected content is displayed on the user's mobile terminal by selecting the desired content displayed on the signage device by operating from the user's mobile terminal. Yes,
Furthermore, a presentation system is also used in which the robot is made to make a predetermined utterance or action in cooperation with a robot having a body similar to a human being and corresponding to the order of the contents to be played.

Japanese Unexamined Patent Publication No. 2015-90600 Japanese Unexamined Patent Publication No. 2015-162149

"Robo Pitch Instruction Manual", [online], [Searched April 3, 2017], Internet [URL: http://www.robo-pitch.com/download/manual.pdf]

However, in order to display and play the content in the set order, the person in charge needs to create the content by deciding in detail the order in which the information contained in the content is played and the playback interval time as well as the content. was there.
In addition, when creating content to be displayed on the signage device or displaying detailed data of the content displayed on the signage device on the mobile terminal by operation from the user's mobile terminal, the user each time It was necessary to perform a predetermined troublesome operation.

In addition, the person in charge gives a presentation while displaying and playing back the contents including the documents created in advance in the set order, and the person in charge performs the input operation by himself / herself to the robot in the order of the contents. It took skill to make a predetermined utterance and movement synchronized, and there were cases where smooth progress could not be achieved.
For example, when the display / playback process of the content and the control process of the robot are not linked, the person in charge of the presentation (hereinafter, also referred to as the operation person) is displaying the predetermined content. , In order for the robot to make a predetermined utterance, etc., it is necessary to input the instruction content such as a character string for uttering to the robot each time. Presentation is difficult.

Alternatively, the person in charge of operation sets in advance the order of the contents used in the presentation, the estimated time required to explain each displayed content, etc., and the utterance executed by the robot so as to correspond to the set contents. If the content and order of the actions and the timing of the execution are set in advance in the robot, it is possible to synchronize the display of the contents with the actions of the robot. However, when actually executing the presentation, the robot must be started at the same time as the presentation starts, and the operator must proceed with the explanation in the presentation according to the set time, and the progress time shifts. In that case, the display of the content and the operation of the robot may not be synchronized, and in order to prevent the synchronization from being out of sync, the operator had to suspend the operation of the robot. ..

That is, it is necessary to set the execution timing of a series of operations executed by the robot in advance according to the playback time interval of the content expected in the presentation performed by humans, but in the actual presentation, the person in charge of operation Since it may not be possible to explain the content at a predetermined time interval, it is difficult to synchronize the reproduction and explanation of the content with the operation of the robot.

On the contrary, it is necessary for the operator who gives the presentation to match the playback and explanation of the content with the execution timing set for the robot, and before giving the presentation, practice the presentation many times and explain the content of the explanation. It is necessary to review and adjust the time of the execution timing of the robot movement, which imposes a heavy burden on the operator in the preparatory stage of the presentation.

In addition, in order to synchronize the playback and explanation of the content used in the presentation with the operation of the robot, the progress of the presentation is monitored from the explanation content of the operator and the content of the playback content, and the progress is supported. It is necessary to develop a special dedicated application program that controls the operation of the robot, and it takes a lot of time and cost to develop it.

Therefore, the present invention has been made in consideration of the above circumstances, and the content creator describes in advance the content of the instruction given to the robot in the content, and when a specific event occurs by the user, a predetermined instruction is given. An object of the present invention is to provide a robot control system capable of easily matching the reproduction of contents used in presentations and the execution timing of robot movements by giving the contents to the robot.

The present invention is a robot control system including a robot and an information processing device that controls the operation of the robot, wherein the information processing device reproduces a content reproduction unit for reproducing the content, the content, and the content. Event information including a storage unit that stores event information in which an event that occurs at that time and a control command for controlling the operation of the robot are associated with each other, and a control command associated with the event when the event occurs. An event execution unit that generates the above, a robot control processing unit that acquires the generated event information and generates transmission information having control contents that the robot can understand from the control commands included in the event information, A first communication unit that transmits transmission information to the robot is provided, and the robot performs a predetermined operation based on the second communication unit that receives the transmission information and the control content included in the received transmission information. It provides a robot control system characterized by having an operation execution unit for executing.

Further, the transmission information transmitted from the information processing device to the robot includes an utterance character string to be spoken by the robot and motion information in which a body motion to be operated by the robot is set. It is characterized by including an audio output unit that outputs an utterance character string included in the received transmission information by synthetic voice, and a body driving unit that makes a gesture motion corresponding to the operation information included in the transmission information. ..

Further, the information processing apparatus stores in advance protocol information, which is a rule for converting predetermined information into information that can be understood by the robot, in the storage unit, and a control command included in the acquired event information is the robot. If the information is incomprehensible, the control command is further provided with an information conversion unit that converts the control command into control contents that can be understood by the robot by using the protocol information, and the first communication unit is the converted control. It is characterized in that transmission information having contents is transmitted to the robot.

Further, the control command is characterized in that it includes a control target for specifying a robot to be controlled and a control content indicating an instruction given to the robot.
Further, the control content indicating the instruction given to the robot included in the control command includes utterance information indicating an utterance character string to be uttered by the robot and operation information for operating the body of the robot. It is a feature.
Further, it is characterized in that a designated variable expressing the utterance content by a predetermined symbol or character is used for the utterance information included in the control content.

Further, the control command is characterized in that it includes a control target that identifies an external control device connected to the robot and a control content for the external control device.
In addition, the event includes an input operation event in which a predetermined selection input is made by the person in charge of operation, a display event in which a predetermined character string is displayed, and a timer event due to the elapse of a predetermined timer setting time. It is characterized by that.

Further, the present invention is an information processing device of a robot control system that controls the operation of a robot, the content reproduction unit for reproducing the content, the content, an event generated when the content is reproduced, and the operation of the robot. A storage unit that stores event information associated with a control command for controlling the above, an event execution unit that generates event information including a control command associated with the event when an event occurs, and the above-mentioned Communication with a robot control processing unit that acquires generated event information and generates transmission information having control contents that the robot can understand from control commands included in the event information, and transmits the transmission information to the robot. It provides an information processing apparatus characterized by having a unit.

Further, the present invention is a control method of a robot control system including a robot and an information processing device for controlling the operation of the robot, wherein the information processing device reproduces a content reproduction step and the content. Among them, when an event occurs, an event execution step that generates event information including a control command associated with the event, and a control command that acquires the generated event information and includes the control command included in the event information. A robot control processing step for generating transmission information having control contents that the robot can understand, and a transmission step for transmitting the transmission information to the robot, and the reception step for the robot to receive the transmission information. Further, the present invention provides a control method for a robot control system, which includes an operation execution step for executing a predetermined operation based on a control content included in the received transmission information.

According to the present invention, in the information processing apparatus, event information in which the content is associated with the event generated when the content is reproduced and the control command for controlling the operation of the robot is stored, for example, the reproduction of the content. When an event occurs, the event information including the control command stored in the setting information is acquired in association with the event, and the control content that the robot can understand is obtained from the control command included in the event information. Since the transmission information to be possessed is generated and the transmission information is transmitted to the robot, it is possible to easily synchronize the reproduction of the content and the timing of causing the robot to operate at the timing intended by the person in charge of operation, and to the robot. Since it is not necessary to set the order and timing of the operations to be executed in advance, it is possible to reduce the burden on the operation person who creates the playback content information.

It is a block diagram of one Example of the robot control system of this invention. It is explanatory drawing of one Example of the information stored in the storage part of the information processing apparatus of this invention. It is explanatory drawing of one Example of the control command stored in the storage part of the information processing apparatus of this invention. It is explanatory drawing of one Example of the control command stored in the storage part of the information processing apparatus of this invention. It is a schematic sequence diagram of one Example such as transmission processing of transmission information which controls a robot of this invention. It is a schematic sequence diagram of one Example such as transmission processing of transmission information which controls a robot of this invention. It is a schematic sequence diagram of one Example such as transmission processing of transmission information which controls a robot of this invention. FIG. 5 is a schematic sequence diagram of an embodiment such as transmission processing of transmission information for controlling an external control device connected to the robot of the present invention. It is a flowchart of one Example such as transmission processing of transmission information in the information processing apparatus of this invention. It is a flowchart of one Example of the operation after receiving the transmission information in the robot of this invention.

Hereinafter, embodiments of the present invention will be described with reference to the drawings. It should be noted that the description of the following examples does not limit the present invention.
<Structure of robot control system>
FIG. 1 shows a block diagram of an embodiment of the robot control system of the present invention.
In FIG. 1, the robot control system of the present invention is mainly composed of an information processing device 1 connected to a network 3 and a robot 2.

Here, the information processing device 1 (hereinafter, also referred to as a PC) is a device that reproduces content generated in advance and controls the operation of the robot 2, and is particularly when displaying predetermined content in a presentation or the like. In addition, it is a device that generates an event for giving a predetermined instruction content to the robot 2 by a user input operation or the like.
The instruction content given to the robot 2 includes, for example, a character string (utterance information) indicating the content to be spoken by the robot 2, and motion information for moving a physical part such as the robot's mouth, eyes, and limbs.
The robot 2 (hereinafter, also referred to as RB) is an autonomous device that performs utterances and body movements corresponding to the instruction contents based on the instruction contents including the utterance information and the operation information given from the information processing device 1.

As the network 3, any existing network may be used, and for example, a wide area network such as the Internet or a LAN used only in a predetermined premises may be used. As the communication method, either wired communication or wireless communication may be used, but since the robot 2 is a device that operates autonomously, it is preferable to be connected to the network by wireless communication.

However, the connection form between the information processing device 1 and the robot 2 is not limited to that via the network 3 as shown in FIG.
For example, the information processing device 1 and the robot 2 may directly connect a communication cable and perform data communication with each other by wired communication without going through the network 3. Alternatively, data communication may be performed with each other by wireless communication using a predetermined wireless signal.

Further, the connection form may be such that the information processing device 1 and the robot 2 perform data communication via the server using the server connected to the network 3.
In this case, the instruction content input by the information processing device 1 includes information for identifying the robot 2 to be controlled, and after the instruction content is transmitted from the information processing device 1 to the server, the server controls the instruction content. The instruction content is transmitted to the robot 2. When the instruction content is transmitted via the server, the server can distribute different instruction content to the plurality of robots 2, or can distribute the same instruction content at the same time.

<Configuration of information processing device>
The information processing device 1 is a device having an information input function and a communication function, and for example, an existing personal computer or a mobile terminal such as a mobile phone may be used.
As shown in FIG. 1, the information processing device 1 mainly includes a control unit 11, a communication unit 12, an information input unit 13, a display unit 14, a content reproduction unit 15, an event execution unit 16, a robot control processing unit 17, and an event monitoring. A unit 18, a control command acquisition unit 19, a transmission information generation unit 20, an information conversion unit 21, and a storage unit 30 are provided.

The control unit 11 is a part that controls the operation of each component such as a communication unit, and is mainly realized by a microcomputer including a CPU, a ROM, a RAM, an I / O controller, a timer, and the like.
The CPU organically operates various hardware based on a control program stored in advance in a ROM or the like to execute the robot control function, the communication function, and the like of the present invention.
In particular, in the present invention, the control unit 11 monitors a program for playing back the content (also referred to as a presentation program), a user's instruction input, or an event that occurs at a predetermined timing, and responds to the event. A robot utilization program for controlling a robot (hereinafter, also referred to as a robot driver) is operated to execute a robot control function or the like.

The communication unit 12 is a part that connects to the network 3 and performs data communication with the robot 2, and corresponds to the first communication unit described above.
For example, the transmission information generated by the robot control processing unit 17 is transmitted to the robot 2. As will be described later, the transmission information is information corresponding to the acquired control command or information after the control command is converted into information that can be understood by the robot.
The connection form to the network 3 may be wired or wireless, and any existing communication procedure may be used.

The information input unit 13 is a part in which a user inputs information such as characters and selects and inputs a function. In the present invention, in particular, for inputting a spoken character string to be spoken by a robot and creating playback content. This is a part for inputting information (for example, text, drawing, image data), inputting control commands for controlling a robot and an external control device, and the like.
As the information input unit 13, for example, a keyboard, a mouse, a touch panel, or the like is used.
Since the content to be spoken by the robot is represented by characters, the user inputs a character string (spoken character string) corresponding to the utterance content, for example, using a keyboard.
In addition, information such as a character string corresponding to the content of the utterance is stored in advance in an external storage medium such as a USB memory, and the storage medium is connected to store the information stored in the storage medium in an information processing device. You may enter in 1. Alternatively, the user may input the utterance content as voice data and perform voice recognition to input a character string (speech character string) corresponding to the utterance content.

Alternatively, a plurality of character string information scheduled to be spoken by the robot is stored in the storage unit 20 in advance, and the stored plurality of character string information is read out and displayed on the display unit 14, and the displayed information is displayed. Of these, by selecting the desired information to be spoken by the robot, it may be input as a character string (spoken character string) corresponding to the uttered content.
In addition, a paper on which the character string to be spoken by the robot is described is created in advance, the character string written on the paper is read as image data by a scanner or a camera, and after character recognition, the character string corresponding to the utterance content (speech). You may enter it as a character string).

The display unit 14 is a part for displaying information, and displays an input character string, information necessary for executing each function, a result of executing the function, and the like. As the display unit 14, for example, an LCD, an organic EL display, or the like is used, and when a touch panel is used as the information input unit 13, the display unit and the touch panel are arranged so as to overlap each other.

The content reproduction unit 15 is a portion for reproducing the content.
For example, the content composed of a plurality of slide images stored in advance in the storage unit 30 is read out, and the contents are reproduced in a predetermined order.
The content to be reproduced may be information stored in advance in the storage unit 30, or may be information stored in advance in a USB memory or other portable storage medium.
In addition, a playback content creation unit that creates information regarding the content to be played back (playback content information) may be provided.

The reproduced content creation unit (not shown) is a part that creates the reproduced content information, and the reproduced content information includes the content to be reproduced.
The content to be reproduced is information intended to be disclosed and reproduced to another person, for example, information used in a presentation, and a plurality of slide images created by a user using the information input unit 13. Is.
Further, the reproduced content information includes, in addition to the content to be reproduced, setting information in which an event generated when the content is reproduced and a control command for controlling the operation of the robot are associated with each other.
The created content and setting information are stored in advance in the storage unit 30.

In the following examples, the content will be described as meaning information consisting of a plurality of slide images. Each slide image is information including documents, drawings, image data, audio data, control commands, and the like. Image data includes still images and moving images.
The control command is information for controlling the operation of the robot and any of the external control devices connected to the robot, as will be described later.

The event execution unit 16 is a part that executes an operation corresponding to an event generated by an input operation or the like of a user (such as a person in charge of operating a presentation).
For example, if an event corresponding to a user's predetermined selection input operation occurs while the presentation program is running, switching of slide images associated with the event in advance, starting video playback, controlling the robot, etc. Executes the processing of.
In particular, when an event related to robot control occurs, event information including a control command stored in the above setting information is generated in association with the event.
The generated event information is mainly given to the robot control processing unit 17.

As will be described later, when the user performs an input operation of clicking a predetermined display area using the mouse while displaying a specific slide image by the presentation program, an event corresponding to the click occurs. do.
If the currently displayed slide image is stored with the event corresponding to the click and the control command indicating the instruction given to the robot associated with each other in advance, it is associated with the event corresponding to the click that has occurred. The control command is sent to the robot.

Specifically, the presentation program mainly corresponds to the event execution unit 16, and when an event corresponding to a click occurs, event information including a control command is given to the robot control processing unit 17.
After that, the transmission information generation unit 20 of the robot control processing unit 17 uses the received control command to generate transmission information to be transmitted to the robot, and the communication unit 12 transmits the generated transmission information to the robot 2. ..

The robot control processing unit 17 is a part that generates transmission information for controlling the robot.
As described above, when the event execution unit 16 generates the event information including the control command, the generated event information is acquired, and the control command included in the event information is used to transmit the control contents that the robot can understand. Generate information.
Further, when an external control device is connected to the robot and the control command included in the event information controls the external control device connected to the robot, transmission for controlling the external control device is performed. Generate information.

The robot control processing unit 17 corresponds to a robot driver, and is mainly composed of an event monitoring unit 18, a control command acquisition unit 19, a transmission information generation unit 20, and an information conversion unit 21.
The event monitoring unit 18 is a unit that monitors the event information given from the event execution unit 16.
The event information output by the event execution unit 16 includes information not related to the control of the robot and information related to the control of the robot, and the event information related to the control of the robot always has a control command. It shall be included.
Therefore, the event monitoring unit 18 monitors whether or not event information related to the control of the robot is given.

The control command acquisition unit 19 is a portion that extracts and acquires the control command included in the event information from the given event information.
When the event monitoring unit 18 detects that the event information related to the control of the robot, that is, the event information including the control command is given, the control command acquisition unit 19 acquires the control command. , Temporarily stored in the storage unit 30.

The transmission information generation unit 20 is a part that generates transmission information to be transmitted to the robot 2 by using a control command temporarily stored in the storage unit 30.
For example, if the control command itself is information in a format that can be understood by the destination robot, the control command itself may be the transmission information.
If the control command itself is not in a format that the destination robot can understand, the information conversion unit 21 converts the control command into information in a format that the destination robot can understand, and the converted information is displayed. Generated as transmission information.

As will be described later, the transmission information transmitted from the information processing device to the robot mainly includes a control target which is information for specifying the destination robot and control contents indicating the contents to be executed by the robot. In addition, the control content includes an utterance character string to be spoken by the robot, motion information in which a body motion to be operated by the robot is set, and the like.

The information conversion unit 21 is a part that converts the control command temporarily stored in the storage unit 30 into information in a format that can be understood by the destination robot.
For this information conversion, the protocol information 35 stored in advance in the storage unit 30 is used.
Protocol information 35 is a rule for converting predetermined information into information that can be understood by a robot.
As described above, when the control command included in the acquired event information is information that the robot cannot understand, the control command is converted into the control content that the robot can understand by using the protocol information 35.
The transmission information having the converted control content is transmitted to the robot by the communication unit 12.

When there are a plurality of formats that the robot can understand, it is preferable to store the protocol information 35 corresponding to all the conversion rules so that the control command can be converted into the information in all the formats.
If the control command is already in a format that the destination robot can understand, it is not necessary to perform this information conversion.
When there are a plurality of destination robots, it is preferable to store in advance a plurality of protocol information 35 for converting the control command into a format that the robot can understand for each robot.

The storage unit 30 is a part that stores information and programs necessary for executing each function of the information processing device of the present invention, and is a storage device such as a semiconductor storage element such as ROM, RAM, or flash memory, or an HDD or SSD. , Other storage media are used.
The storage unit 30, for example, stores playback content information 31, event information 32, control command 33, transmission information 34, protocol information 35, and the like.
FIG. 2 shows an explanatory diagram of an embodiment of information stored in the storage unit of the information processing apparatus of the present invention.

The reproduced content information 31 is information composed of documents, drawings, images, sounds, etc. used in presentations, bulletin boards, etc., and the content is reproduced by displaying the information on a display device or a screen. For example, when giving a presentation, a plurality of slide images correspond to playback content information, and the person in charge of the presentation slides while displaying the plurality of slide images in order or selecting the slide images to be displayed. The contents of the image will be explained.
The slide image includes not only a still image but also multimedia information such as a document, a drawing, a moving image, and an audio.

FIG. 2A shows an explanatory diagram of an embodiment of the reproduced content information 31.
Here, in the case of giving a presentation, playback content information composed of a plurality of slide images is shown. For example, the content CN01 is composed of five slide images (SL01 to SL05), and similarly, the contents CN02 and CN03 are shown to be composed of a plurality of slide images.

In addition, when the robot is made to perform utterances, limbs, and other gestures while the content is being played, it is necessary to associate the slide image to be played with the information for controlling the robot in advance. ..
The information for controlling the robot is a control command described later, and some control commands are created separately from the slide image and stored in the storage unit 30 in advance.
In the control command, the operation content to be executed by the robot is set in a predetermined description format.
For example, when the robot is made to speak, the control command is set to speak as an instruction action, and the utterance character string is set.

In addition, in order to determine the timing at which the control command is transmitted to the robot when the operator in charge of the presentation sets in advance the association between the slide image used in the presentation and the control command in which the operation to be executed by the robot is set. Also set to associate events such as input operations with control commands.
At this time, setting information is created in which the event that occurs when the content is played and the control command for controlling the operation of the robot are associated with each other.

For example, if the person in charge of operation performs an input operation of clicking a predetermined display area of a slide image with a mouse or the like and wants to send a control command to the robot, the control command is input with a click. Make settings to associate.
The input operation for determining the timing of sending this control command to the robot is called an event. For example, an input operation called click is called a click event.
Generally, when a predetermined event such as a click event occurs, event information (also called an event message) is generated, but in the present invention, a setting for associating a control command with an event is made. By doing so, the control command will be included in the generated event information.

For example, by setting in advance a setting for associating a slide image, a control command for causing a robot to speak, and a click event, an input in which an operator clicks a predetermined display area while playing the slide image. When a click event occurs after an operation, event information including a control command associated with the click event is generated, and transmission information corresponding to the control command included in the event information is transmitted to the robot. ..
When the robot receives the transmission information, it executes the utterance set in the control command corresponding to the transmission information.

Events that occur when playing content include, for example, an input operation event in which a predetermined selection input is made by the operator, a display event in which a predetermined character string is displayed on the display unit, and a predetermined timer setting time. Includes timer events due to the passage of time.
The click event is one of the input operation events by the person in charge of operation.

FIG. 2B shows an explanatory diagram of the associated slide image, the event, and an embodiment of the control command for the content CN01 of the reproduced content information 31.
Here, it is assumed that the content CN01 is composed of five slide images (SL01 to SL05), and each slide image is stored as a display file (g1 to g5).
For example, the slide image SL01 is a document on which characters and charts are described, and when the slide image SL01 is reproduced, the display file g1 is read out and displayed on the display unit 14.
Further, the slide image SL03 is a moving image, and when the slide image SL03 is played back, the display file g3 is read out and the moving image is displayed on the display unit 14.

In FIG. 2B, it is assumed that the setting information in which the event content and the control command are associated with each slide image is stored.
The control command is information indicating the execution operation of the robot associated with the slide image, and the event content means a causal event (event) that triggers the transmission of the control command to the robot.
When the display file is read and the slide image is being played, when the cause event (event) set in the event content associated with the slide image occurs, the robot is notified by the associated control command. The content of the operation to be executed is transmitted.

For example, the slide image SL01 indicates that the click event EV01 is set as the event content, and the information CMD01 that causes the robot to speak is set in advance as the control command.
By this association, in the playback state of the display file g1 of the slide image SL01, when the click event EV01 indicating that the operator clicks the predetermined display area is generated, the robot is made to speak. The command CMD01 indicates that it will be sent to the robot.

Further, in the slide image SL02, it is shown that the timer event EV02 is set as the event content, and the information CMD02 for causing the robot to make a gesture operation is set in advance as the control command.
In this case, in the playback state of the display file g2 (still image) of the slide image SL02, for example, when the timer is started after the still image is displayed and the time preset for the timer elapses, the timer event EV02 It is indicated that the control command CMD02, which determines that the occurrence has occurred and causes the robot to perform a gesture motion, is transmitted to the robot.

Further, the slide image SL03 indicates that the click event EV03 is set as the event content, and the information CMD03 that causes the robot to speak and make a gesture is set in advance as a control command.
In this case, when the click event EV03 indicating that the operator has performed an input operation to click the predetermined display area occurs in the playback state of the slide image SL03 display file g3 (video), the robot is uttered. It indicates that the control command CMD03 for gesturing is transmitted to the robot.

Further, the slide image SL04 indicates that the display event EV04 is set as the event content, and the information CMD04 for causing the robot to make a gesture operation is set in advance as the control command.
In this case, when the predetermined character string is displayed in the playback state of the display file g4 (document) of the slide image SL04, it is determined that the display event EV04 has occurred, and the control command CMD04 that causes the robot to make a gesture is executed. Indicates that it will be sent to the robot.

Further, in the slide image SL05, the click event EV05 is set as the event content, but nothing is set as the control command for controlling the robot.
In this case, in the playback state of the slide image SL05 display file g4 (still image), processing for the robot occurs when a click event EV05 indicating that the operator has performed an input operation to click a predetermined display area occurs. It means that the process other than is executed, but the process for the robot is not executed because nothing is set as the control command.

Note that FIG. 2B shows an example in which one control command is associated with each slide image, but the present invention is not limited to this.
For example, a plurality of control commands may be associated with one slide image and set.
In this case, when the event content associated with the slide image occurs, the associated control commands are activated in the set order, and the robot is made to perform operations corresponding to the multiple control commands in sequence. You may do so.

Further, a plurality of event contents and control commands may be set in association with one slide image. In this case, when an event content associated with any of the plurality of control commands occurs in advance, the control command is transmitted to the robot to execute the operation set in the control command.

FIG. 2B shows an example in which one event content and control command are set in association with each slide image, but the setting information consisting of one event content and control command is not associated with the slide image. May be set to.
For example, apart from the slide image of the content, the information of the part including the event content and the control command shown in FIG. 2B is set in advance.
That is, a setting is made to associate the control command that describes the content to be operated by the robot with the event that triggers the transmission of the control command to the robot, and the setting information is stored in the storage unit 30 in advance.

In this case, the robot can be made to execute an operation corresponding to a desired control command set in advance regardless of the slide image being reproduced.
For example, when the robot wants to perform some action, a list consisting of a plurality of event contents and control commands set in advance is displayed on the display unit, and the operator selects a desired control command from the list. Operate. After that, when an event associated with the selected control command occurs, the control command is transmitted to the robot to cause the robot to perform a desired operation.
Thereby, during the reproduction of an arbitrary slide image, the person in charge can give the robot a desired instruction content to the robot by performing an operation of selecting a desired control command, and the slide image used for the presentation. It is possible to easily match the reproduction of the robot with the execution timing of the robot operation.

The event information 32 is information generated by the event execution unit 16 when a predetermined event occurs, and as described above, the event information 32 including the control command is transmitted from the event execution unit 16 to the robot control processing unit 16. Given to 17.

FIG. 2C shows an explanatory diagram of an embodiment of event information 32.
Here, as the event information 32, the event content and the control command are shown.
When the information shown in FIG. 2B is preset, for example, when the click event EV01 is generated by the person in charge performing a click operation while the slide image SL01 is being played, FIG. From the information shown in 2 (b), the control command CMD01 corresponding to the click event EV01 is detected, and event information 32 including the generated click event EV01 and the detected control command CMD01 is generated.

Further, when a predetermined timer event EV02 occurs while the slide image SL02 shown in FIG. 2 (b) is being played, the control command CMD02 corresponding to the timer event EV02 is issued from the information shown in FIG. 2 (b). Event information 32 including the detected and generated click event EV02 and the detected control command CMD02 is generated.
The event information 32 generated by the event execution unit 16 is given to the robot control processing unit 17.

However, the event information 32 is not limited to the one shown in FIG. 2 (c), and various information is generated. As the event information for controlling the robot, as described above, the event information including the control command is generated, but in the case of the event not controlling the robot, the event information not including the control command is generated.

FIG. 2D shows an explanatory diagram of an embodiment of an event used for robot control.
Here, four events of "click", "animation", "display", and "timer" are illustrated, but the present invention is not limited to these.
FIG. 2D shows an event name, an abbreviation, and a brief description of the event for each event.
For example, the click event "click CL" is an event meaning that when a user performs a predetermined selection input operation, the selection input operation gives an instruction to activate a predetermined operation.

Further, the animation event "animation AM" is an event meaning that when the user performs a predetermined selection input operation, the user is instructed to activate the predetermined animation by the selection input operation.
Further, the display event "display DS" is an event meaning that when a predetermined character string is displayed on the display unit, an instruction to activate a predetermined operation is given as a trigger of the display.
In addition, a timer event called "timer TM" is an event that means that when the time set for the activated timer elapses, an instruction to activate a predetermined operation is given as a trigger of the elapse of the time. ..

The control command 33 is information for controlling the robot, and is, for example, created in advance by the person in charge of operating the presentation and stored in the storage unit 30.
The control command 33 includes at least a control target 33a and a control content 33b.
3 and 4 show explanatory views of an embodiment of a control command stored in the storage unit of the information processing apparatus of the present invention.
The control target 33a is information that identifies a control robot that transmits a control command, and is, for example, an identification number (robot number RBID) preset for each robot.

The control content 33b is information indicating an instruction given to the robot, and includes, for example, a designated operation, operation information, and utterance information.
The designated motion is information that identifies the motion to be executed by the robot, and is information that identifies, for example, utterance, body motion, image shooting, voice recognition, device control, and the like.
The motion information is information for operating the body of the robot, and is information for setting the meaning and content of the motion specified by the designated motion.

The utterance information is information indicating an utterance character string to be uttered by the robot, and is also information in which a character string to be recognized by the robot is set in the voice uttered by a human.
As the utterance information included in the control content 33b, a designated variable expressing the utterance content by a predetermined symbol or character may be used as described later.

Further, in the control command 33, an external control device 80 connected to the robot may be set as the control target 33a, and the control content 33b for the external control device 80 may be included. When the control command 33 related to the external control device 80 is transmitted to the robot, the operation of the external control device 80 connected to the robot is controlled.

FIG. 3A shows an embodiment of the information included in the control command.
In FIG. 3A, robo01 is set as the robot number RBID which is the control target 33a, sp which means utterance is set as the designated operation which is the control content 33b, and a predetermined basic operation is set as the operation information. 1 is set, which means to do, and the utterance character string "Good morning" is set as the utterance information.
When this control command is transmitted to the robot whose robot number is robo01, the robot outputs a synthetic voice of "Good morning" and executes a preset basic operation such as a gesture.

FIG. 3B shows an embodiment of the control command description format.
In order to include the information shown in FIG. 3 (a) in the control command, the description format as shown in FIG. 3 (b) is used. This description format is an example and is not limited to this.
In FIG. 3B, the control command is described in the form of [Control target: Designated operation] Control operation content.
FIG. 3B illustrates two control commands.
The control command CMD01 is information consisting of [robo01: sp] 1 and "good morning", which describes the control command shown in FIG. 3 (a).

In addition, the control command CMD02 is information consisting of [robo02: qa] "Good morning" and "How are you?", And is information given to the robot of robo02, which is the control target. This is an instruction operation that means that the robot recognizes the voice and further speaks the described character string.
Here, the first written character string "Good morning" corresponds to the recognition character string, and the second written character string "How are you?" Corresponds to the utterance character string.
When this control command CMD02 is sent to the robot whose robot number is robo02, the robot recognizes whether or not the voice "Good morning" is input, and recognizes that the voice "Good morning" is input. If so, a synthetic voice saying "How are you?" Is output.

When the control command in the description format shown in FIG. 3B is data in a format that can be understood by the robot, the control command in this description format may be transmitted to the robot as it is, and the robot receiving the control command may receive the control command. , The contents of the control command can be interpreted and the operation corresponding to the control command can be executed immediately.

However, if the control command in the description format shown in FIG. 3B is data in a format that the robot cannot understand, even if the control command in this description format is transmitted to the robot as it is, the desired operation cannot be performed. .. Therefore, as described above, the information conversion unit 21 converts the control command in the description format shown in FIG. 3B into transmission information in a format that can be understood by the robot.
The transmitted information after conversion is transmitted to the robot.
For this information conversion, the protocol information 35 stored in advance in the storage unit 30 is used.

FIG. 3 (c) shows a description example of some control commands.
FIG. 3C shows a description example of the six control commands and a description of the description example.
For example, the control command CMD01 is information consisting of [robo01: sp] 0 and "good morning". This means that the control target is a robot called robo01, the specified action is utterance, and the utterance information is "good morning". It means that. Further, 0 is set as the motion information, which means that there is no gesture motion to be executed by the robot.
Therefore, when the robot named robo01 receives this control command CMD01, it outputs the voice of the character string "Good morning" without gesturing.

The control command CMD02 is information consisting of [robo02: sp] 1 and "good morning". The control target is a robot called robo02, the specified movements are utterance and gesture movements, and the utterance information is "good morning". , Which means that the preset basic operation 1 is executed as the operation information.
Therefore, when the robot named robo02 receives this control command CMD02, it outputs the voice corresponding to the character string "Good morning" while executing the gesture operation corresponding to the basic operation 1.

The control command CMD03 is information consisting of [robo03: qa] "aaa" and "bbb". The control target is a robot called robo03, the specified operation is character recognition and utterance, and the recognition character string is. It is "aaa", which means that the utterance character string is "bbb".
Therefore, when the robot named robo03 receives this control command CMD03, it outputs the voice of the character string "bbb" when it recognizes the voice of the recognition character string "aaa".

The control command CMD04 is information consisting of [robo04: cam], which means that the control target is a robot called robo04 and the specified operation is camera shooting.
Therefore, when the robot named robo04 receives this control command CMD04, the camera connected to the robot takes an image.

The control command CMD05 is information consisting of [dev01: on], which means that the control target is the external control device dev01 connected to the robot, and the specified operation turns on the power of the external control device dev01. It means to start it.
Therefore, when the robot to which the external control device dev01 is connected receives this control command CMD05, the power of the external control device dev01 connected to the robot is turned on and started.
When there are a plurality of robots to be controlled, information for designating a robot to which an external control device is connected and information for designating an external control device are set as control targets.

The control command CMD06 is information consisting of [dev02: off], which means that the control target is the external control device dev02 connected to the robot, and the specified operation turns off the power of the external control device dev02. It means to stop the operation.
Therefore, when the robot to which the external control device dev02 is connected receives this control command CMD06, the power of the external control device dev02 connected to the robot is turned off and the operation is stopped.
In addition to the control commands shown in FIG. 3 (c), various control commands may be defined.

FIG. 4 shows an embodiment of information (control target, designated operation, operation information, utterance information) of four items included in the control command.
FIG. 4A shows an embodiment of the control target 33a included in the control command.
The control target 33a is information for specifying a device that can be controlled by a control command, and includes, for example, a robot (robo) and an externally connected device (dev) connected to the robot.
A robot identification number such as robo01 is used to identify the robot to be controlled.
A device identification number, such as dev01, is used to identify an externally connected device connected to the robot.
However, the device that can be controlled is not limited to the above, and any device that can directly communicate with the information processing device 1, understands the control command, and can perform a predetermined operation is sufficient, and is physically connected to the robot. It may be a device that has not been processed.

FIG. 4B shows an embodiment of the designated operation included in the control command.
The designated operation is information that identifies the operation to be executed by the robot and the externally connected device connected to the robot.
For a robot, it is information that identifies a function that the robot can perform. For example, a robot has a speech function (voice output function), body movement function such as limbs, voice recognition function, image shooting function, character recognition function, image recognition function such as face, autonomous driving function, projection function, display function, application. This is information that specifies which function to execute when it has an execution function of.

FIG. 4B shows four operations as designated operations to be executed by the robot.
When sp is described as the designated operation, the robot is instructed to execute the utterance function. The character string to be spoken by the robot is described in the control operation content part of the control command shown in FIG. 3 (b).

When qa is described as the designated operation, the robot is instructed to execute the voice recognition and utterance functions. The voice to be recognized by the robot and the character string to be spoken by the robot when the robot recognizes the voice to be recognized are described in the control operation content part of the control command shown in FIG. 3 (b).

When sel is described as the designated operation, the robot is instructed to execute a plurality of voice recognition and utterance functions. A plurality of voices to be recognized by the robot and a plurality of character strings to be spoken by the robot when the robot recognizes the voices to be recognized are described in the control operation content part of the control command shown in FIG. 3 (b). NS. The plurality of recognized voices and the spoken character strings are described in association with each other so that a specific character string is spoken with respect to the recognized voice.

When cam is described as the designated operation, it is instructed to execute the function of taking an image with the camera connected to the robot.
Information does not have to be described in the control operation content part of the control command.
However, when a still image or a moving image is specified as the image to be captured, the distinction between the still image or the moving image may be described in the control operation content part of the control command.
Further, when a person is included in the captured image and the face of the person is recognized, it may be described in the control operation content part of the control command that the face of the person should be recognized. Alternatively, it may be described that the result of face recognition should be spoken in a predetermined voice.

FIG. 4B shows two operations as designated operations to be executed by the externally connected device connected to the robot.
For example, when on is described as the specified operation, it is instructed to turn on the power of the externally connected device and start it.
In addition, when off is described as the designated operation, it is instructed to turn off the power of the externally connected device and stop it.
Externally connected devices connected to the robot include, for example, lighting equipment, display devices, speakers, microphones, home appliances (televisions, air conditioners, watches, etc.), cars, and the like.
The designated operation shown in FIG. 4B is not limited to these, and other operations may be set.

FIG. 4C shows an embodiment of the operation information included in the control command.
The motion information is information that specifies a gesture motion to be executed by the robot. For example, a gesture motion such as a limb of the robot that is performed at the same time as an utterance is designated.
FIG. 4 (c) shows three types of operation contents at the time of utterance, and as the operation information at the time of utterance, a number corresponding to any of the three types of operation contents is assigned to the control operation contents part of the control command. Describe.

For example, when 0 is described as the motion information at the time of utterance, it means that the gesture motion is not executed at the same time as the utterance.
When 1 is described as the operation information at the time of utterance, it means that the preset basic operation is executed at the same time as the utterance.
When 2 is described as the operation information at the time of utterance, it means that the operation corresponding to the preset designated number is executed at the same time as the utterance.
However, the operation information included in the control command is not limited to that shown in FIG. 4 (c).

FIG. 4D shows an embodiment of utterance information included in the control command.
The utterance information indicates the content that causes the robot to speak by voice.
As the utterance information, a character string that means the content itself to be uttered is mainly described, but as shown in FIG. 4 (d), information indicating the time or a fixed form is a designated variable using a predetermined symbol or character. You may specify a character string or the like.
FIG. 4D shows the designated variables of the information indicating the five times.

For example, when the specified variable (day) is described in the control operation content part of the control command, it means that the robot is made to speak the current date.
In addition, when the designated variable (dayofweek) is described in the control operation content part of the control command, it means that the robot is made to speak today's day of the week.
Further, when the designated variable (hour) is described in the control operation content part of the control command, it means that the robot is made to speak the current time.
Further, when the designated variable (minute) is described in the control operation content part of the control command, it means that the robot is made to speak the current minute.
Further, when the designated variable (second) is described in the control operation content part of the control command, it means that the robot is made to speak the current second.

As described above, the transmission information 34 is the information generated by the transmission information generation unit 20 and is the information transmitted to the robot.
Similar to the control command, the transmission information 34 includes at least a control target 34a and a control content 34b.
When the robot can understand the control command in the description format as shown in FIG. 3 and can make a predetermined utterance or operation based on the control command, the acquired control command 33 is transmitted to the robot as it is. In this case, the transmission information 34 is the same data as the control command 33.
However, when the control command is described in a format that the robot cannot understand, it is necessary to change the control command 33 into a predetermined format that the robot can understand and transmit it as described above. In this case, the information after the control command 33 is converted by the protocol information 35 becomes the transmission information 34.

The protocol information 35 is information that defines conversion rules for converting the control command 33 into a predetermined format that can be understood by the robot.
For example, when a standardization specification is set as data for controlling a robot and the destination robot can understand the data corresponding to this standardization specification, the control command 33 is used as the protocol information 35 in the standardization specification. The conversion rule for converting to the data corresponding to is stored.

As described above, the information conversion unit 21 converts the content of the control command 33 into data in a format conforming to the standardization specifications by using the protocol information 35. The data in a format conforming to the standardization specifications becomes the transmission information 34 and is transmitted to the robot RB.
Note that the protocol information 35 is stored in the robot RB in advance, the control command 33 that the robot cannot understand is transmitted from the information processing device PC to the robot RB, and the control command 33 received by the robot RB is stored. The protocol information 35 may be used to convert the information into information that the robot itself can understand.

The above is the configuration of the information processing device 1. When a server is used as the robot control system, for example, among the configurations of the information processing device 1 described above, the functions of the transmission information generation unit 20 and the information conversion unit 21 The information processing device 1 may transmit a control command and information for identifying a destination robot to the server.
In this case, the server may generate transmission information in a format that the robot can understand from the received control command, and transmit the transmission information to the robot.

<Robot configuration>
As shown in FIG. 1, the robot 2 mainly includes a control unit 51, a communication unit 52, an output voice generation unit 53, an output operation determination unit 54, an operation execution unit 55, a voice output unit 56, a body drive unit 57, and a voice input. A unit 58, an image input unit 59, a device control unit 60, and a storage unit 70 are provided.

The control unit 51 is a part that controls the operation of each component such as a communication unit, and is mainly realized by a microcomputer including a CPU, a ROM, a RAM, an I / O controller, a timer, and the like.
The CPU organically operates various hardware based on a control program stored in advance in a ROM or the like to execute the audio output function, the communication function, and the like of the present invention.

The communication unit 52 is a part that connects to the network 3 and performs data communication with the information processing device 1, and corresponds to the second communication unit described above.
For example, as described above, the transmission information 34 generated by the information processing device 1 is received.
The received transmission information 34 is stored in the storage unit 70 as the reception information 71.
The connection form to the network 3 may be either wired or wireless, but in the case of a robot that autonomously travels, it is preferable to use wireless, and any existing communication procedure may be used.
When receiving information for controlling the robot via the server, data communication is performed with the server.

The output voice generation unit 53 is a part that generates voice data to be output from the speaker, which is the voice output unit 56. The utterance content to be output is determined based on the transmission information transmitted from the information processing device 1.
The utterance character string included in the transmission information 34 becomes the utterance content that is actually output as voice, and a synthetic voice of the utterance content is generated.
The generated synthetic voice is stored in the storage unit 70 as the output voice information 72.

The output motion determination unit 54 is a portion that determines a gesture motion using the limbs of the robot based on the transmission information 34 transmitted from the information processing device 1.
For example, the gesture motion related to the meaning of the utterance content included in the received transmission information 34 and the motion information is determined by using the information of the basic motion stored in the storage unit 70 in advance.
The determined gesture motion is stored in the storage unit 70 as output motion information 73.

The operation execution unit 55 is a unit that executes a predetermined operation based on the control content included in the received transmission information 34, and mainly includes a voice output unit 56 and a body drive unit 57.
The voice output unit 56 is a part that outputs synthetic voice generated by the output voice generation unit 53, and corresponds to a speaker. For example, the utterance character string included in the received transmission information 34 is output by synthetic voice.

The body drive unit 57 is a portion that executes a gesture motion determined by the output motion determination unit 54. For example, the gesture operation corresponding to the operation information included in the received transmission information 34 is performed.
When the robot has a human-like body, it changes the facial expression of the robot and moves the neck, arms, limbs, and torso.

The voice input unit 58 is a microphone for inputting voice. When the robot has a voice recognition function, it recognizes the voice input by the microphone, converts it into characters, and understands the meaning and content of the voice.
The image input unit 59 is a camera for inputting an image, and captures a still image or a moving image. When the robot has an image recognition function, it recognizes an object photographed by a camera, for example, a person's face.

The device control unit 60 is a part that controls the external control device 80.
The external control device 80 is a device having a function of accepting remote control, and includes, for example, lighting equipment, display devices, home appliances (televisions, air conditioners, watches, etc.), cars, and the like.
When the received transmission information 34 includes control information meaning device control, a predetermined control signal is transmitted to the external control device 80 based on the control information.
For example, a control signal for turning on the power is transmitted or a control signal for turning off the power is transmitted to the external control device 80 such as a lighting fixture or a display device.
The voice input unit 58, the image input unit 59, and the device control unit 60 do not have to have essential configurations, and may be added as optional functions as needed.

The storage unit 70 is a part that stores information and programs necessary for executing each function of the robot of the present invention, and is a semiconductor storage element such as ROM, RAM, flash memory, a storage device such as HDD, SSD, and the like. Storage medium is used.
The storage unit 70 stores, for example, received information 71, output voice information 72, output operation information 73, and the like.

The received information 71 here means the transmission information 34 sent from the information processing device 1.
Therefore, the received information 71 includes at least the control target 71a and the control content 71b.
Using the received information 71, the output voice generation unit 53 generates a synthetic voice, and the output operation determination unit 54 determines the gesture operation.

The output voice information 62 is a synthetic voice generated by the output voice generation unit 53.
The output motion information 63 is information generated by the output motion determination unit 54, and is information that sets the movements of physical parts such as limbs, mouth, and eyes provided in the robot.

<Explanation of Examples of Robot Control in Robot Control System>
Here, mainly in the information processing device PC, the person in charge creates the playback content information and the control command indicating the instruction content to the robot RB in advance, and the person in charge of operation performs the robot while giving a presentation. An embodiment of a series of processes for causing the robot to perform a predetermined operation including utterance by performing an input operation or the like for giving an instruction to the RB is shown.

The person in charge of operation creates in advance a plurality of control commands having the description format as shown in FIG. 3 in which the utterances and body movements to be executed by the robot are set. Further, for each of the created plurality of control commands, a setting for associating a causal event (event content) that gives the control command to the robot and the control command is set and stored in the storage unit 30 in advance. Alternatively, as shown in FIG. 2B, the setting information associated with the event content and the control command is set to associate with each slide image constituting the content, and the storage unit is stored as the reproduced content information 31. Store in 30 in advance.

When the person in charge of operation actually gives a presentation, the application program (presentation program) for executing the presentation is started.
In addition, when the person in charge performs an input operation to specify the playback content information used for the presentation and the setting information associated with the event content and the control command, the presentation program starts playing the content, for example, the content. The first slide image of the above is displayed on the display unit 14, and the presentation is ready.
When the content played by the presentation program controls the robot, the robot is used, which corresponds to the robot control processing unit 17 that controls the robot after processing for connecting the information processing device PC and the robot RB. Start the program (robot driver).

(First Example)
FIG. 5 shows a schematic sequence diagram of an embodiment such as transmission information transmission processing for controlling the robot of the present invention.
In the first embodiment, an operator who connects one information processing device PC and one robot RB and gives a presentation starts a presentation using the PC and during the presentation. , The case where the person in charge of operation performs a click operation corresponding to an event input for controlling the robot will be described.

In FIG. 5, it is assumed that the process of connecting the information processing device PC and the robot RB has already been completed, and the presentation program and the robot utilization program (robot driver) corresponding to the robot control processing unit 17 have been started. do.

In step A1 of FIG. 5, the person in charge of operation specifies the playback content information used for the presentation and the setting information associated with the event content and the control command by the information input unit 13 of the PC, and starts playing the content. Suppose you have performed an input operation that means.

In step P1, when the control unit 11 of the PC confirms that the input meaning the start of playback of the content has been input, the designated playback content information 31 is read from the storage unit 30 and the playback of the content is started. For example, the display unit 14 displays the first slide image included in the read content. The playback of this content is started by the presentation program.
After that, the person in charge of operation gives a presentation based on the materials prepared in advance while using the displayed slide image.

Next, in step A2, it is assumed that the person in charge of operation performs a click operation in order to make the robot perform a predetermined operation during the presentation.
It is assumed that this click operation is an event input related to the control of the robot. For example, if there is a display area associated with the robot in the slide image currently displayed in the presentation, the person in charge of operation moves the cursor to this display area and clicks with the mouse.

In step P2, when the control unit 11 of the PC confirms this click operation, it detects that an event input (click event) related to the control of the robot has occurred. By detecting this click event, the presentation program corresponding to the event execution unit 16 uses the setting information associated with the event content of the playback content information and the control command to display the generated event content and the event content. The event information 32 including the associated control command is generated and given to the robot utilization program (robot driver) corresponding to the robot control processing unit 17.

For example, as shown in FIG. 2B, if the click event EV01 occurs when the slide image SL01 included in the content CN01 of the playback content information 31 is displayed, as shown in FIG. 2C. Event information 32 including the generated event content EV01 and the control command CMD01 associated with the click event EV01 is generated.
As the control command CMD01, for example, as shown in Fig. 3 (c), when [robo01: sp] 0, "Good morning" is set, [robo01: sp] 0, "Good morning" is the event information. 32 is included.

In step P3, the event monitoring unit 18 of the robot control processing unit 17 monitors the event information 32, and when it detects that the event information 32 has been given, the event information 32 is acquired.
Further, in step P4, the control command acquisition unit 19 of the robot control processing unit 17 acquires the control command 33 included in the event information 32.
When the above control command CMD01 is included in the event information 32, [robo01: sp] 0, "Good morning" is acquired as the control command 33.

In step P5, the transmission information generation unit 20 generates the transmission information 34 by using the acquired control command 33. When the acquired control command 33 is information that can be understood by the robot, the control command itself becomes the transmission information 34.
If the acquired control command 33 is information that the robot cannot understand, the information conversion unit 21 converts the control command 33 into information in a format that the robot can understand, and then transmits the converted information. Generate as 34.

In step P6, the transmission information 34 generated by the communication unit 12 is transmitted to the robot RB.

In step R1, the communication unit 52 of the robot RB receives the transmission information 34 and stores it in the storage unit 70 as the reception information 71.
In step R2, the control unit 51 of the robot RB analyzes the contents of the received information 71 and determines the operation to be executed.

In step R3, when the action to be executed is utterance, the output voice generation unit 53 extracts the utterance character string included in the received information 71, generates the synthetic voice to be uttered, and stores it as the output voice information 72. ..
When the motion to be executed is the body motion of the robot, the output motion determination unit 54 determines the content of the body motion such as limbs and stores it as the output motion information 73.

In step R4, perform the operation requested by the PC.
For example, the voice output unit 56 outputs synthetic voice based on the output voice information 72.
Further, the body driving unit 57 executes the determined body movements such as limbs based on the output movement information 73.
The output of the synthetic voice and the body movements such as limbs may be performed at different timings, but when the utterance and the body movements are related, the output of the synthetic voice and the body movements such as limbs are performed at almost the same timing. Is preferable.

By the above processing, when the operator intends to make the robot perform a predetermined motion during the presentation, the operator inputs an event (for example, click) associated with the control command describing the predetermined motion in advance. The robot can be made to perform a predetermined operation just by performing an event).

Since the timing for causing the robot to perform a predetermined operation is set by an input operation of the operator, for example, a click, the operator is in charge of the operation even if the progress of the presentation is earlier or later than planned. The robot can be made to execute a predetermined operation at the most appropriate timing by the input operation of the person himself / herself, and the reproduction of the content used in the presentation and the timing at which the robot is made to perform the predetermined operation are easily synchronized. be able to.

In the present invention, in order to synchronize the reproduction of the content with the operation to be executed by the robot, the presentation program for controlling the reproduction of the content is subjected to a predetermined operation to be executed by the robot in addition to the processing of the reproduction of the content. It is not necessary to preset the contents together with the order and timing for executing the predetermined operation.
As described above, in the presentation program, the event content and control as shown in FIGS. 2 and 3 so that the operator can determine the timing of the operation to be executed by the robot according to the progress of the presentation. Since it is only necessary to set and store the information for robot control associated with the command, it is possible to reduce the burden on the operator when creating the operation to be executed by the robot.

(Second Example)
FIG. 6 shows a schematic sequence diagram of an embodiment such as transmission information transmission processing for controlling the robot of the present invention.
In this second embodiment, an operator who connects one information processing device PC and a plurality of robots RBs and gives a presentation starts a presentation using the PC and during the presentation. , The case where the person in charge of operation performs a click operation corresponding to an event input for controlling each of a plurality of robots will be described.
Although FIG. 6 shows three robot RBs, it may be configured to connect two or four or more robots RBs.

Further, as shown in FIG. 6, it is assumed that the three setting information in which the event content and the control command are associated are stored in advance in the playback content information, and the event occurs according to the description order of the three setting information. Then, it is assumed that the transmission information corresponding to the control command associated with the event is transmitted to the robot.
For ease of explanation, the event contents of the three setting information are all click operations performed by the person in charge of operation.

In FIG. 6, it is assumed that the process of connecting the information processing device PC and each robot RB has already been completed, and the presentation program and the robot utilization program (robot driver) corresponding to the robot control processing unit 17 have been started. do.
Since the processing of steps P2 to P6 and the processing of steps R1 to R4 shown in FIG. 5 are performed on the PC and RB, a part of the steps and the description is omitted.

In step P11 of FIG. 6, it is assumed that the person in charge of operation performs an input operation meaning the start of playback of the content, the content is read from the storage unit, and the playback of the content is started.
For example, the display unit 14 displays the first slide image included in the read content. After that, the person in charge of operation gives a presentation based on the materials prepared in advance while using the displayed slide image.

Next, in step A11, it is assumed that the person in charge of operation performs a click operation in order to make the robot perform a predetermined operation during the presentation.
This click operation is the same input operation as the click operation shown in FIG. 5 of the first embodiment.

After that, although omitted in FIG. 6, when this click operation is confirmed on the PC, as shown in FIG. 5, it is detected that an event input (click event) has occurred, and event information 32 is generated. , And the event monitoring unit 18 of the robot control processing unit 17 acquires the event information 32.

Further, in step P12, the control command acquisition unit 19 acquires the control command 33 included in the event information 32.
As described above, assuming that the three setting information in which the event content and the control command in FIG. 6 are associated with each other are set so that the event occurs according to the description order of the three setting information, the first click operation is performed. If this is done, event information 32 including the control command ([robo01: sp] 1, character string 01) associated with the click 01 described first is generated, so the control command ([robo01: sp]] 1, character string 01) is acquired. Since the control target of this control command is robo01, the transmission information is transmitted to the robot robo01.

The transmission information generation unit 20 generates the transmission information 34 by using the acquired control command 33, and in step P13, the transmission information 34 generated by the communication unit 12 is transmitted to the robot robo01.
Here, the transmission information corresponding to the control command ([robo01: sp] 1, character string 01) is transmitted to the robot robo01.
As described above, when the acquired control command 33 is information that can be understood by the robot robo01, the control command itself becomes the transmission information 34, and when the information is information that the robot robo01 cannot understand, the information conversion unit 21 After the control command 33 is converted into information in a format that can be understood by the robot robo01, the converted information becomes the transmission information 34.

In step R11, the communication unit 52 of the robot robot 01 receives the transmission information 34 and stores it in the storage unit 70 as the reception information 71.
After that, the content of the received information 71 is analyzed to determine the operation to be executed.
When [robo01: sp] 1 and the character string 01 are received as the reception information 71, it is determined that the actions to be executed are the utterance and the setting basic action 1.
Further, the output voice generation unit 53 extracts the character string 01 included in the received information 71, generates a synthetic voice to be spoken from the character string 01, and stores it as the output voice information 72. Further, as the body movement to be executed by the robot, the content of the setting basic movement 1 is stored as the output movement information 73.

In step R12, the voice output unit 56 outputs the synthetic voice of the character string 01 based on the output voice information 72. Further, the body driving unit 57 executes body movements such as limbs corresponding to the setting basic movement 1 based on the output movement information 73.
The output of the synthetic voice and the execution of the body movement may be performed in parallel at the same time.

First, in the processing up to this point, the operation person plays the content, for example, while displaying a predetermined slide image, the operation person performs the first click operation at a desired timing. The transmission information corresponding to the first control command associated with the first click operation is transmitted to the controlled robot robo01 set in the control command, and is intended at the intended timing. It is possible to make the robot robot01 perform the movement.

After that, the person in charge of operation continues the presentation while displaying other slide images prepared in advance.
Then, during the presentation, it is assumed that the person in charge of operation determines that it is time to cause the robot to perform a predetermined operation.
At this time, in step A12, the person in charge of operation performs a second click operation in order to cause the robot to perform a predetermined operation.

After that, when the second click operation is confirmed on the PC, the event input (click event) is detected as in the case of confirming the first click operation, and the event information 32 is generated. It is given to the robot control processing unit 17, and the event monitoring unit 18 of the robot control processing unit 17 acquires the event information 32.

Further, in step P14, the control command acquisition unit 19 acquires the control command 33 included in the event information 32.
As described above, assuming that the events are set to occur according to the description order of the three setting information in FIG. 6, when the second click operation is performed, it is associated with the second click 02 described. Since the event information 32 including the control command ([robo02: sp] 2, character string 02) is generated, the control command ([robo02: sp] 2, character string 02) is acquired. Since the control target of this control command is robo02, the transmission information is transmitted to the robot robo02.

The transmission information generation unit 20 generates the transmission information 34 by using the acquired control command 33, and in step P15, the transmission information 34 generated by the communication unit 12 is transmitted to the robot robo02.
Here, the transmission information corresponding to the control command ([robo02: sp] 2, character string 02) is transmitted to the robot robo02.

In step R21, the communication unit 52 of the robot robot 02 receives the transmission information 34, stores it in the storage unit 70 as the reception information 71, analyzes the contents of the reception information 71, and determines the operation to be executed.
When [robo02: sp] 2 and the character string 02 are received as the reception information 71, it is determined that the actions to be executed are the utterance and the designated number action 2.
Further, the output voice generation unit 53 extracts the character string 02 included in the received information 71, generates a synthetic voice to be spoken from the character string 02, and stores it as the output voice information 72. Further, as the body movement to be executed by the robot, the content of the designated number movement 2 is stored as the output movement information 73.

In step R22, the voice output unit 56 outputs the synthetic voice of the character string 02 based on the output voice information 72. Further, the body driving unit 57 executes body movements such as limbs corresponding to the designated number movement 2 based on the output movement information 73.

By the processing up to this point, for example, while displaying a predetermined slide image, the person in charge of operation performs the second click operation at a desired timing, so that the second click operation is associated in advance. The transmission information corresponding to the second control command is transmitted to the robot robot02 to be controlled, which is set in the control command, and the intended operation can be caused to the robot robot02 at the intended timing. ..

Further, the person in charge of operation continues the presentation while displaying another slide image or the like prepared in advance.
Then, during the presentation, when the operator determines that it is time to cause the robot to perform a predetermined motion, in step A13, the operator performs a third click operation in order to cause the robot to perform the predetermined motion. do.

After that, when the PC confirms this third click operation, it detects that an event input (click event) has occurred and generates event information 32, as in the case of confirming the previous click operation. It is given to the robot control processing unit 17, and the event monitoring unit 18 of the robot control processing unit 17 acquires the event information 32.

Further, in step P16, the control command acquisition unit 19 acquires the control command 33 included in the event information 32.
As described above, assuming that the events are set to occur according to the description order of the three setting information in FIG. 6, when the third click operation is performed, it is associated with the third click 03 described. Since the event information 32 including the control command ([robo03: sp] 0, character string 03) is generated, the control command ([robo03: sp] 0, character string 03) is acquired. Since the control target of this control command is robo03, the transmission information is transmitted to the robot robo03.

The transmission information generation unit 20 generates the transmission information 34 by using the acquired control command 33, and in step P17, the transmission information 34 generated by the communication unit 12 is transmitted to the robot robo03.
Here, the transmission information corresponding to the control command ([robo03: sp] 0, character string 03) is transmitted to the robot robo03.

In step R31, the communication unit 52 of the robot robo03 receives the transmission information 34, stores it in the storage unit 70 as the reception information 71, analyzes the contents of the reception information 71, and determines the operation to be executed.
When [robo03: sp] 0 and the character string 03 are received as the received information 71, it is determined that the only action to be executed is utterance and no physical action is performed.
Further, the output voice generation unit 53 extracts the character string 03 included in the received information 71, generates a synthetic voice to be spoken from the character string 03, and stores it as the output voice information 72. There is no physical movement to be executed by the robot, and nothing is stored in the output movement information 73.

In step R32, the voice output unit 56 outputs the synthetic voice of the character string 03 based on the output voice information 72.

By the processing up to this point, for example, while displaying a predetermined slide image, the person in charge of operation performs the third click operation at a desired timing, so that the third click operation is associated in advance. The transmission information corresponding to the third control command is transmitted to the controlled robot robo03 set in the control command, and the intended utterance can be made to the robot robo03 at the intended timing. ..

Further, in this embodiment, it is assumed that the three setting information in which the event content and the control command are associated are set so that the event occurs according to the description order of the three setting information. Select the action to be executed by the robot before each click operation. Instead of performing the input operation, simply perform the simple input operation of clicking, and let multiple robots perform the desired actions in a preset order. be able to.

(Third Example)
FIG. 7 shows a schematic sequence diagram of an embodiment such as transmission information transmission processing for controlling the robot of the present invention.
In the third embodiment, as in the second embodiment, an operator who connects one information processing device PC and a plurality of robots RB and gives a presentation uses the PC to give a presentation. A case where the person in charge of operation performs a click operation corresponding to an event input for controlling each of a plurality of robots will be described. FIG. 7 shows three robots RB.

Further, as shown in FIG. 7, it is assumed that the three setting information in which the event content and the control command are associated are stored in advance in the playback content information. The control command corresponding to each event content is the same as that shown in the second embodiment of FIG.

However, unlike the second embodiment of FIG. 6, it is assumed that the description order of the three setting information associated with the event content and the control command does not match the event occurrence order, and the person in charge of operation determines the robot. If you want to make the operation of, select the control command corresponding to the operation, and then click.
That is, in the third embodiment, regardless of the description order of the setting information in which the event contents and the control commands are associated with each other, the person in charge of operation operates the desired operation to be executed by the robot anytime, any number of times. It can be executed at any timing intended by the person in charge.

Also in FIG. 7, the process of connecting the information processing device PC and each robot RB has already been completed, and the presentation program and the robot utilization program (robot driver) corresponding to the robot control processing unit 17 have been started. And.
Since the processing of steps P2 to P6 and the processing of steps R1 to R4 shown in FIG. 5 are performed on the PC and RB, a part of the steps and the description is omitted.

In step P21 of FIG. 7, it is assumed that the person in charge of operation performs an input operation meaning the start of playback of the content, the content is read from the storage unit 30, and the playback of the content is started.
For example, the display unit 14 displays the first slide image included in the read content. After that, the person in charge of operation gives a presentation based on the materials prepared in advance while using the displayed slide image.

Next, in step A21, during the presentation, the operator performs an operation of selecting a control command corresponding to the operation in order to cause the robot to perform a predetermined operation, and then performs a click operation. And.
Here, first, three setting information in which the event content and the control command as shown in FIG. 7 are associated with each other are displayed on the display unit 14, and the person in charge of operation causes the robot among the displayed control commands. All you have to do is select the control command that corresponds to the operation you want to perform.

In step A21, it is assumed that the control command corresponding to the click 03 is selected from the three setting information in which the event content displayed on the display unit 14 and the control command are associated with each other.
In addition, instead of separating the operation to select the control command and the click operation, it is assumed that the operation to select the control command includes the click operation, and the click event is generated just by performing the operation to select the control command. May be regarded as.

After that, when the PC confirms this click operation, it detects that an event input (click event) has occurred, generates event information 32, and causes the robot control processing unit 17 to generate event information 32, as shown in FIG. Then, the event monitoring unit 18 of the robot control processing unit 17 acquires the event information 32.

Further, in step P22, the control command acquisition unit 19 acquires the control command 33 included in the event information 32.
As described above, if the operation of selecting the control command corresponding to the click 03 is performed in the three setting information associated with the event content and the control command in FIG. 7, the control command associated with the click 03 ( Event information 32 including [robo03: sp] 0, character string 03) is generated, and a control command ([robo03: sp] 0, character string 03) is acquired. Since the control target of this control command is robo03, the transmission information is transmitted to the robot robo03.

The transmission information generation unit 20 generates the transmission information 34 by using the acquired control command 33, and in step P23, the transmission information 34 generated by the communication unit 12 is transmitted to the robot robo03.
Here, the transmission information corresponding to the control command ([robo03: sp] 0, character string 03) is transmitted to the robot robo03.

In step R33 and step R34, the same processing as in step R31 and step R32 of FIG. 6 is performed.
That is, in step R33, the communication unit 52 of the robot robo03 receives the transmission information 34, stores it in the storage unit 70 as the reception information 71, analyzes the contents of the reception information 71, and determines the operation to be executed.
When [robo03: sp] 0 and the character string 03 are received as the received information 71, it is determined that the only action to be executed is utterance and no physical action is performed.

Further, the output voice generation unit 53 extracts the character string 03 included in the received information 71, generates a synthetic voice to be spoken from the character string 03, and stores it as the output voice information 72. There is no physical movement to be executed by the robot, and nothing is stored in the output movement information 73.
In step R32, the voice output unit 56 outputs the synthetic voice of the character string 03 based on the output voice information 72.

After that, the person in charge of operation continues the presentation while displaying other slide images prepared in advance, and during the presentation, the person in charge of operation determines that it is time for the robot to perform a predetermined operation. And.

At this time, in step A22, three setting information in which the event content and the control command as shown in FIG. 7 are associated with each other are displayed on the display unit 14, and the person in charge of operation displays the three setting information displayed. Of these, it is assumed that the click operation is performed after the operation of selecting the control command corresponding to the click 01 is performed.

After that, when the PC confirms this click operation, it detects that an event input (click event) has occurred, generates event information 32, and causes the robot control processing unit 17 to generate event information 32, as shown in FIG. Then, the event monitoring unit 18 of the robot control processing unit 17 acquires the event information 32.

Further, in step P24, the control command acquisition unit 19 acquires the control command 33 included in the event information 32.
As described above, if the operation of selecting the control command corresponding to the click 01 is performed, the event information 32 including the control command ([robo01: sp] 1, character string 01) associated with the click 01 is generated. , The control command ([robo01: sp] 1, character string 01) is acquired. Since the control target of this control command is robo01, the transmission information is transmitted to the robot robo01.

The transmission information generation unit 20 generates the transmission information 34 by using the acquired control command 33, and in step P25, the transmission information 34 generated by the communication unit 12 is transmitted to the robot robo01.
Here, the transmission information corresponding to the control command ([robo01: sp] 1, character string 01) is transmitted to the robot robo01.

In step R13 and step R14, the same processing as in step R11 and step R12 of FIG. 6 is performed.
That is, in step R13, the communication unit 52 of the robot robo01 receives the transmission information 34, stores it in the storage unit 70 as the reception information 71, analyzes the contents of the reception information 71, and determines the operation to be executed.
When [robo01: sp] 1 and the character string 01 are received as the reception information 71, it is determined that the actions to be executed are the utterance and the setting basic action 1.

Further, the output voice generation unit 53 of the robot robot 01 extracts the character string 01 included in the received information 71, generates a synthetic voice to be spoken from the character string 01, and stores it as the output voice information 72. Further, as the body movement to be executed by the robot, the content of the setting basic movement 1 is stored as the output movement information 73.

In step R14, the voice output unit 56 of the robot robot 01 outputs the synthetic voice of the character string 01 based on the output voice information 72. Further, the body drive unit 57 of the robot robo01 executes body movements such as limbs corresponding to the setting basic movement 1 based on the output movement information 73.
The output of the synthetic voice and the execution of the body movement may be performed in parallel at the same time.

After that, the operator similarly continues the presentation while displaying other slide images prepared in advance, and during the presentation, the timing at which the operator causes the robot to perform a predetermined operation is set. Suppose you decide that you have come.

At this time, in step A23, three setting information in which the event content and the control command as shown in FIG. 7 are associated with each other are displayed on the display unit 14, and the person in charge of operation displays the three setting information displayed. Of these, it is assumed that the click operation is performed after the operation of selecting the control command corresponding to the click 02 is performed.

After that, when the PC confirms this click operation, it detects that an event input (click event) has occurred, generates event information 32, and causes the robot control processing unit 17 to generate event information 32, as shown in FIG. Then, the event monitoring unit 18 of the robot control processing unit 17 acquires the event information 32.

Further, in step P26, the control command acquisition unit 19 acquires the control command 33 included in the event information 32.
As described above, if the operation of selecting the control command corresponding to the click 02 is performed, the event information 32 including the control command ([robo02: sp] 2, character string 02) associated with the click 02 is generated. , The control command ([robo02: sp] 2, character string 02) is acquired.

The transmission information generation unit 20 generates the transmission information 34 by using the acquired control command 33, and in step P27, the transmission information 34 generated by the communication unit 12 is transmitted to the robot robo02.
Here, the transmission information corresponding to the control command ([robo02: sp] 2, character string 02) is transmitted to the robot robo02.

In step R23 and step R24, the same processing as in step R21 and step R22 of FIG. 6 is performed.
That is, in step R23, the communication unit 52 of the robot robot 02 receives the transmission information 34, stores it in the storage unit 70 as the reception information 71, analyzes the contents of the reception information 71, and determines the operation to be executed.
When [robo02: sp] 2 and the character string 02 are received as the reception information 71, it is determined that the actions to be executed are the utterance and the designated number action 2.
Further, the output voice generation unit 53 extracts the character string 02 included in the received information 71, generates a synthetic voice to be spoken from the character string 02, and stores it as the output voice information 72. Further, as the body movement to be executed by the robot, the content of the designated number movement 2 is stored as the output movement information 73.

In step R24, the voice output unit 56 outputs the synthetic voice of the character string 02 based on the output voice information 72. Further, the body driving unit 57 executes body movements such as limbs corresponding to the designated number movement 2 based on the output movement information 73.

Further, in the subsequent presentation, for example, when it is desired to execute the operation corresponding to the click 01, the person in charge of the operation associates the event content as shown in FIG. 7 with the control command as in step A22. The three setting information may be displayed on the display unit 14, and the control command corresponding to the click 01 may be selected from the three displayed setting information, and then the click operation may be performed.
As a result, the robot robo01 can be made to perform the operation associated with the click 01 again.

In the third embodiment, the setting information in which the event content and the control command are associated is displayed on the display unit 14, and the desired control command is selected from the displayed setting information, and then the click operation is performed. Therefore, the operator can execute the desired operation to be executed by the robot at any time, any number of times, at any timing intended by the operator.

(Fourth Example)
FIG. 8 shows a schematic sequence diagram of an embodiment such as transmission information transmission processing for controlling an external control device connected to the robot of the present invention.
In the fourth embodiment, as in the first embodiment, an operator who connects one information processing device PC and one robot robo01 and gives a presentation uses the PC to give a presentation. And during the presentation, the person in charge of operation corresponds to the click operation corresponding to the event input for controlling the robot and the event input for controlling the external control device dev01 connected to the robot robo01. The case where the click operation is performed will be described.

In FIG. 8, it is assumed that the robot robo01 and the external control device dev01 are connected in advance, and the operation of the external control device dev01 is controlled by the control signal transmitted from the robot robo01. Further, since there is only one robot connected to the PC, it is assumed that all the transmission information transmitted from the PC is transmitted to the robot robo01.

As shown in FIG. 8, it is assumed that the three setting information in which the event content and the control command are associated are stored in advance in the playback content information.
Here, the control command associated with the click 01, which is the event content, is the information that sets the operation for the robot, and the control command associated with the click 02 and the click 03 sets the operation for the external control device. It shall be information.

Also in FIG. 8, as in the third embodiment of FIG. 7, it is assumed that the description order of the three setting information associated with the event content and the control command does not match the event occurrence order, and the person in charge of operation If you want the robot to perform a predetermined action, you shall perform an operation to select a control command corresponding to that action, and then perform a click operation.
However, as in the second embodiment, an event may occur in accordance with the description order of the three setting information, and transmission information corresponding to the control command associated with the event may be transmitted.

Further, also in FIG. 8, the process of connecting the information processing device PC and the robot RB has already been completed, and the presentation program and the robot utilization program (robot driver) corresponding to the robot control processing unit 17 have been started. It shall be.
Since the processing of steps P2 to P6 and the processing of steps R1 to R4 shown in FIG. 5 are performed on the PC and RB, a part of the steps and the description is omitted.

In step P31 of FIG. 8, it is assumed that the person in charge of operation performs an input operation meaning the start of playback of the content, the content is read from the storage unit 30, and the playback of the content is started.
For example, the display unit 14 displays the first slide image included in the read content. After that, the person in charge of operation gives a presentation based on the materials prepared in advance while using the displayed slide image.

Next, in step A31, three setting information in which the event content and the control command as shown in FIG. 8 are associated with each other are displayed in order to cause the robot to perform a predetermined operation during the presentation. It is assumed that the display is displayed on the unit 14 and the person in charge of operation selects the control command corresponding to the click 01 from the displayed control commands, and then performs the click operation.

After that, when the PC confirms this click operation, it detects that an event input (click event) has occurred, generates event information 32, and causes the robot control processing unit 17 to generate event information 32, as shown in FIG. Then, the event monitoring unit 18 of the robot control processing unit 17 acquires the event information 32.

Further, in step P32, the control command acquisition unit 19 acquires the control command 33 included in the event information 32.
As described above, if the operation of selecting the control command corresponding to the click 01 is performed, the event information 32 including the control command ([robo01: sp] 1, character string 01) associated with the click 01 is generated. , The control command ([robo01: sp] 1, character string 01) is acquired.

The transmission information generation unit 20 generates the transmission information 34 by using the acquired control command 33, and in step P33, the transmission information 34 generated by the communication unit 12 is transmitted to the robot robo01.
Here, the transmission information corresponding to the control command ([robo01: sp] 1, character string 01) is transmitted to the robot robo01.

In step R51 and step R52, the same processing as in step R11 and step R12 of FIG. 6 is performed.
That is, in step R51, the communication unit 52 of the robot robot 01 receives the transmission information 34, stores it in the storage unit 70 as the reception information 71, analyzes the contents of the reception information 71, and determines the operation to be executed.
When [robo01: sp] 1 and the character string 01 are received as the reception information 71, it is determined that the actions to be executed are the utterance and the setting basic action 1.

Further, the output voice generation unit 53 of the robot robot 01 extracts the character string 01 included in the received information 71, generates a synthetic voice to be spoken from the character string 01, and stores it as the output voice information 72. Further, as the body movement to be executed by the robot, the content of the setting basic movement 1 is stored as the output movement information 73.

In step R52, the voice output unit 56 of the robot robot 01 outputs the synthetic voice of the character string 01 based on the output voice information 72. Further, the body drive unit 57 of the robot robo01 executes body movements such as limbs corresponding to the setting basic movement 1 based on the output movement information 73.
The output of the synthetic voice and the execution of the body movement may be performed in parallel at the same time.

After that, the person in charge of operation continues the presentation while displaying other slide images prepared in advance, and during the presentation, the person in charge of operation operates the external control device dev01 connected to the robot robo01. It is assumed that it is determined that it is time to control the control.
First, it is assumed that the power of the external control device dev01 is turned on and started.

In this case, in step A32, in order to turn on the power of the external control device dev01, the display unit 14 displays three setting information in which the event content and the control command as shown in FIG. 8 are associated with each other, and the operation is performed. The person in charge performs an operation of selecting a control command corresponding to click 02 from the displayed control commands, and then performs a click operation.

After that, when the PC confirms this click operation, it detects that an event input (click event) has occurred, generates event information 32, gives it to the robot control processing unit 17, and monitors the event of the robot control processing unit 17. The unit 18 acquires the event information 32.

Further, in step P34, the control command acquisition unit 19 acquires the control command 33 included in the event information 32.
As described above, if the operation of selecting the control command corresponding to the click 02 is performed, the event information 32 including the control command [dev01: on] associated with the click 02 is generated, and the control command [dev01: on] is generated. ] Is acquired.

The transmission information generation unit 20 generates the transmission information 34 by using the acquired control command 33, and in step P35, the transmission information 34 generated by the communication unit 12 is transmitted to the robot robo01.
Here, the transmission information corresponding to the control command [dev01: on] is transmitted to the robot robo01.

In step R53, the communication unit 52 of the robot robot 01 receives the transmission information 34, stores it in the storage unit 70 as the reception information 71, analyzes the contents of the reception information 71, and determines the operation to be executed.
When [dev01: on] is received as the reception information 71, the control target is the external control device dev01 connected to the robot robo01, and the operation to be executed for the external control device dev01 is to turn on the power. Is determined to be.

In step R54, the device control unit 60 of the robot robo01 transmits a control signal (startup signal) meaning that the power is turned on to the external control device dev01.
In step D01, when the external control device dev01 receives the control signal (startup signal) meaning that the power is turned on, it executes the process of turning on the power itself to start a predetermined operation.

After that, it is assumed that the person in charge of operation continues the presentation and determines that it is time to turn off the power of the external control device dev01 connected to the robot robo01 and stop the operation during the presentation.

In this case, in step A33, in order to turn off the power of the external control device dev01, the display unit 14 displays three setting information in which the event content and the control command as shown in FIG. 8 are associated with each other, and the operation is performed. The person in charge performs an operation of selecting a control command corresponding to the click 03 from the displayed control commands, and then performs a click operation.

After that, when the PC confirms this click operation, it detects that an event input (click event) has occurred, generates event information 32, gives it to the robot control processing unit 17, and monitors the event of the robot control processing unit 17. The unit 18 acquires the event information 32.

Further, in step P36, the control command acquisition unit 19 acquires the control command 33 included in the event information 32.
As described above, if the operation of selecting the control command corresponding to the click 03 is performed, the event information 32 including the control command [dev01: off] associated with the click 03 is generated, and the control command [dev01: off] is generated. ] Is acquired.

The transmission information generation unit 20 generates the transmission information 34 by using the acquired control command 33, and in step P37, the transmission information 34 generated by the communication unit 12 is transmitted to the robot robo01.
Here, the transmission information corresponding to the control command [dev01: off] is transmitted to the robot robo01.

In step R55, the communication unit 52 of the robot robot 01 receives the transmission information 34, stores it in the storage unit 70 as the reception information 71, analyzes the contents of the reception information 71, and determines the operation to be executed.
When [dev01: off] is received as the received information 71, the control target is the external control device dev01 connected to the robot robo01, and the operation to be executed for the external control device dev01 is to turn off the power. Is determined to be.

In step R56, the device control unit 60 of the robot robo01 transmits a control signal (stop signal) meaning that the power is turned off to the external control device dev01.
In step D02, when the external control device dev01 receives the control signal (stop signal) meaning that the power is turned off, the process of turning off the power is executed to stop the predetermined operation.

In the fourth embodiment, the information for controlling the operation of the external control device connected to the robot is described in the setting information in which the event content and the control command are associated with each other, and the person in charge of operation describes the event content. The setting information associated with the control command is displayed on the display unit 14, and the control command for controlling the operation of the external control device is selected from the displayed information, and then the click operation is performed. The operation of the external control device can be easily controlled via the robot simply by performing the same operation as the operation for controlling the operation of the robot.
That is, it is not necessary to memorize different types of input operations in order to control the operation of the external control device, and the operation load of the operator for controlling the operation of the external control device can be reduced.

<Explanation of processing flow of information processing device in robot control system>
FIG. 9 shows a flowchart of an embodiment such as transmission processing of transmission information in the information processing apparatus of the present invention.
In the information processing device PC, the reproduced content information 31 and the control command 33 as described above are created and stored in the storage unit 30, and the control command and the event content that defines the timing of transmitting the control command to the robot are associated with each other. It is assumed that the set setting information is stored in the storage unit 30. This flow substantially corresponds to steps P1 to P6 of the information processing apparatus PC in the sequence shown in FIG.

In step S1 of FIG. 9, a robot utilization program (robot driver) corresponding to the robot control processing unit 17 is activated.
In step S2, the process of connecting the information processing device PC and the robot RB is performed.
The communication unit of the PC and the communication unit of the RB confirm that both are in a communicable state by communicating predetermined information with each other based on a predetermined communication procedure for connection.

In step S3, the presentation program corresponding to the event execution unit 16 is started, and the reproduction content used in the presentation is read out from the reproduction content information 31 stored in the storage unit 30.

In step S4, playback of the read playback content is started. For example, the first slide image included in the content is displayed on the display unit 14.
After that, the person in charge of operation gives a presentation using the displayed image or document.
During a presentation, when the person in charge of operation clicks on a specific display area of the displayed slide image, for example, a click event occurs. Alternatively, a display event occurs when a predetermined character string is displayed on the slide image.
Here, the occurrence of a predetermined event such as the occurrence of a click event or the occurrence of a display event is referred to as event input.

In step S5, check whether there is an event input.
If it is detected in step S6 that there is an event input, the process proceeds to step S8, and if not, the process proceeds to step S7.
In step S7, the reproduction process of the reproduced content is continued as it is, and the process returns to step S5. do. For example, when there are a plurality of slide images and it is desired to display another slide image, the person in charge of operation performs an operation of switching the slide images and proceeds with the presentation.

When it is detected that the event has been input, the presentation program generates the event information 32 in step S8.
If there is a control command associated with the content of the event that has occurred in the setting information associated with the event content and the control command stored in advance in the storage unit 30, the event information 32 including the control command. Is generated.

In step S9, the event monitoring unit 18 of the robot control processing unit 17 monitors the generation of the event information 32 and acquires the event information 32.
In step S10, it is checked whether or not the acquired event information 32 includes a control command for the robot.
In step S11, if the control command is included, the process proceeds to step S12, and if not, the process proceeds to step S7.

In step S12, the control command acquisition unit 19 acquires the control command 33 included in the event information 32.
In step S13, the transmission information generation unit 20 generates the transmission information 34 by using the acquired control command 33.

As described above, when the control command 33 is information that can be understood by the robot, the control command 33 may be used as the transmission information 34 as it is. Alternatively, when the control command 33 is information that the robot cannot understand, the information conversion unit 21 uses the protocol information 35 to convert the control command 33 into information that the robot can understand, and transmits the converted information. Information 34 may be used.
In step S14, the communication unit 12 transmits the generated transmission information 34 to the robot. After that, the process returns to step S5 and the above process is repeated.

<Explanation of robot processing flow in robot control system>
FIG. 10 shows a flowchart of an embodiment such as an operation after receiving transmission information in the robot of the present invention.
The processing when the external control device is connected to the robot RB is shown below, but steps S55, S61, and S62 are not necessary when the external control device is not connected to the robot RB.

In step S51 of FIG. 10, a process of connecting the robot RB and the information processing device PC is performed.
In step S52, the reception check of the data transmitted from the information processing apparatus PC is performed.
If the transmission information transmitted from the information processing apparatus PC is received in step S53, the process proceeds to step S54, and if not, the process returns to step S52.

In step S54, the received transmission information is stored as the reception information 71, the reception information 71 is analyzed, the control target is specified from the control command included in the reception information 71, and the operation to be executed is performed from the control content of the control command. To judge.
If the determined operation in step S55 is the control of the external control device, the process proceeds to step S61, and if not, the process proceeds to step S56.

In step S61, a control signal for controlling the external control device is generated based on the control content of the control command.
In step S62, the generated control signal is transmitted to the external control device 80.
Then, the process returns to step S52.

If the determined action is an utterance in step S56, the utterance character string is acquired from the received information.
In step S57, the output voice generation unit 53 generates a synthetic voice to be spoken from the acquired utterance character string and stores it as the output voice information 72.
In step S58, the output motion determination unit 53 determines the content of the body motion such as limbs from the control content of the control command and stores it as the output motion information 73.

In step S59, the voice output unit 56 outputs the synthesized voice by using the output voice information 72.
In step S60, the body driving unit 57 drives each part of the body of the robot by using the output motion information 73 to execute a predetermined gesture motion.
The audio output in step S59 and the gesture operation in S60 may not be executed in this order, but may be executed in conjunction with each other at substantially the same timing.
Then, the process returns to step S52.

1 Information processing device,
2 robot,
3 networks,
11 Control unit,
12 Communication Department,
13 Information input section,
14 Display,
15 Content playback department,
16 Event Execution Department,
17 Robot control processing unit,
18 Event Monitoring Department,
19 Control command acquisition unit,
20 Transmission information generator,
21 Information conversion unit,
30 Memory
31 Playback content information,
32 Event information,
33 control commands,
33a Control target,
33b Control content,
34 Transmission information,
34a Control target,
34b Control content,
35 Protocol information,
51 Control unit,
52 Communication Department,
53 Output voice generator,
54 Output operation determination unit,
55 Action Execution Unit,
56 Audio output unit,
57 Body drive,
58 Voice input section,
59 Image input section,
60 Equipment control unit,
70 storage,
71 Received information,
71a Control target,
71b Control content,
72 Output audio information,
73 Output operation information,
80 External control equipment

Claims (10)

A robot control system consisting of a robot and an information processing device that controls the operation of the robot.
The information processing device
The content playback unit that plays content and the content playback section
A storage unit that stores in advance playback content information including the content, an event that occurs when the content is played, and setting information associated with a control command for controlling the operation of the robot.
When an event occurs during playback of the predetermined content, a control command associated with the event that has occurred is acquired by using the setting information included in the playback content information, and the generated event and the event occur. an event executing unit for generating event information including the acquired control command,
A robot control processing unit that acquires the generated event information and generates transmission information having control contents that the robot can understand from the control commands included in the event information.
It is provided with a first communication unit that transmits the transmission information to the robot.
The robot
The second communication unit that receives the transmission information and
A robot control system including an operation execution unit that executes a predetermined operation based on the control content included in the received transmission information. The transmission information transmitted from the information processing device to the robot includes an utterance character string to be spoken by the robot and motion information in which a body motion to be operated by the robot is set.
The motion execution unit of the robot
An audio output unit that outputs the utterance character string included in the received transmission information by synthetic voice, and
The robot control system according to claim 1, further comprising a body driving unit that performs a gesture motion corresponding to the motion information included in the transmission information. The information processing device stores in advance protocol information, which is a rule for converting predetermined information into information that can be understood by the robot, in the storage unit.
When the control command included in the acquired event information is information that the robot cannot understand, an information conversion unit that converts the control command into control contents that the robot can understand by using the protocol information is further provided. ,
The robot control system according to claim 1, wherein the first communication unit transmits transmission information having the converted control content to the robot. The robot control system according to any one of claims 1 to 3, wherein the control command includes a control target for specifying a robot to be controlled and a control content indicating an instruction given to the robot. The control content that indicates the instruction given to the robot included in the control command is characterized in that it includes utterance information indicating an utterance character string to be uttered by the robot and motion information for operating the body of the robot. The robot control system according to claim 4. The robot control system according to claim 5, wherein a designated variable expressing the utterance content by a predetermined symbol or character is used for the utterance information included in the control content. The robot control according to any one of claims 1 to 3, wherein the control command includes a control target that identifies an external control device connected to the robot and a control content for the external control device. system. The event includes an input operation event in which a predetermined selection input is made by the person in charge of operation, a display event in which a predetermined character string is displayed, and a timer event due to the elapse of a predetermined timer set time. The robot control system according to any one of claims 1 to 7. An information processing device for a robot control system that controls the movement of a robot.
The content playback unit that plays content and the content playback section
A storage unit that stores in advance playback content information including the content, an event that occurs when the content is played, and setting information associated with a control command for controlling the operation of the robot.
When an event occurs during playback of the predetermined content, a control command associated with the event that has occurred is acquired by using the setting information included in the playback content information, and the generated event and the event occur. an event executing unit for generating event information including the acquired control command,
A robot control processing unit that acquires the generated event information and generates transmission information having control contents that the robot can understand from the control commands included in the event information.
An information processing device including a communication unit that transmits the transmission information to the robot. It is a control method of a robot control system consisting of a robot and an information processing device that controls the operation of the robot.
The information processing device
Content playback steps and content playback steps
When an event occurs during playback of the content, an event that occurs when the content included in the playback content information stored in advance is played is associated with a control command for controlling the operation of the robot. An event execution step that acquires a control command associated with an event that has occurred using the setting information and generates event information including the generated event and the acquired control command, and an event execution step.
A robot control processing step that acquires the generated event information and generates transmission information having control contents that the robot can understand from the control commands included in the event information.
It includes a transmission step of transmitting the transmission information to the robot.
The robot
A receiving step for receiving the transmission information and
A control method for a robot control system, which comprises an operation execution step for executing a predetermined operation based on a control content included in the received transmission information.

Images