WO2020032249A1

WO2020032249A1 - Information processing device, mediation device, simulate system, and information processing method

Info

Publication number: WO2020032249A1
Application number: PCT/JP2019/031615
Authority: WO
Inventors: 康彦橋本; 掃部　雅幸; 繁次田中; 佳彦丸山
Original assignee: 川崎重工業株式会社
Priority date: 2018-08-10
Filing date: 2019-08-09
Publication date: 2020-02-13
Also published as: CN112543960A; KR20210043621A; EP3836126A4; TWI744676B; KR102571997B1; TW202017626A; EP3836126A1; US11908337B2; US20210291369A1

Abstract

This information processing device (100) converts first information for manipulating a robot model into second information for manipulating a robot model of a simulate device (200, 40, 50) which executes simulation for causing the robot model to perform a simulated operation, the first information having been input into a manipulating terminal (20) which is connected to the simulate device (200, 40, 50) and is manipulated by a remote user of the simulate device (40, 50). The simulate device (40, 50) is operated in accordance with the second information, and the manipulating terminal (20) is caused to present information about the operation of the robot model of the simulate device (200, 40, 50) operating in accordance with the second information.

Description

Information processing device, mediation device, simulation system, and information processing method

The present invention relates to an information processing device, an intermediary device, a simulation system, and an information processing method.

作業 Conventionally, human-made operations are being converted to robots. When the robot is operated by a human, training of human operation is required. For example, Patent Document 1 discloses a training device for operating a life support robot arm. This training device is realized by a simulator that is executed on a computer device and simulates the function of a life support robot arm.

JP-A-2011-125991

In recent years, in industries handling industrial robots, there is a concern that there will be a shortage of robot operators due to the declining birthrate and aging population. On the other hand, for some reason, the number of people who cannot work or work sufficiently is increasing. The effort of such people is precious.

Therefore, an object of the present invention is to provide an information processing device, an intermediary device, a simulation system, and an information processing method that enable various people to perform a robot operation by simulating the operation of the robot. I do.

In order to achieve the above object, an information processing device according to one embodiment of the present invention is connected to a simulation device that performs a simulation for causing a robot model to perform a simulated operation, and is operated by a remote user of the simulation device. The first information for operating the robot model, which is input to the operation terminal to be converted, is converted into second information for operating the robot model of the simulation device, and the simulation device is operated according to the second information. And causing the operation terminal to present information on the operation of the robot model of the simulation device that operates according to the second information.

The mediation device according to one aspect of the present invention includes the information processing device according to one aspect of the present invention, and mediates the operation terminal and the simulation device via a communication network.

In addition, the simulation system according to one embodiment of the present invention includes an information processing device according to one embodiment of the present invention, the simulation device, and an intermediary that mediates the operation terminal and the simulation device via a communication network. Device.

Further, the information processing method according to one aspect of the present invention is an information processing method which is connected to a simulation device that executes a simulation for causing a robot model to perform a simulated operation, and is input to an operation terminal operated by a remote user of the simulation device. The converted first information for operating the robot model is converted to second information for operating the robot model of the simulation device, and operating the simulation device according to the second information, Information on the operation of the robot model of the simulation device that operates according to the two information is presented to the operation terminal.

According to the present invention, various people can operate the robot by simulating the operation of the robot.

FIG. 1 is a diagram illustrating an example of a configuration of a simulation system according to an embodiment. FIG. 2 is a block diagram illustrating an example of a hardware configuration of the operation terminal according to the embodiment. FIG. 3 is a block diagram illustrating an example of a hardware configuration of the mediation device according to the embodiment. FIG. 4 is a block diagram illustrating an example of a hardware configuration of the simulation computer according to the embodiment. FIG. 5 is a block diagram illustrating an example of a hardware configuration of the simulation robot according to the embodiment. FIG. 6 is a block diagram illustrating an example of a functional configuration of the operation terminal according to the embodiment. FIG. 7 is a block diagram illustrating an example of a functional configuration of the intermediary device according to the embodiment. FIG. 8 is a diagram illustrating an example of a simulation selection screen presented by the operation terminal according to the embodiment. FIG. 9 is a block diagram illustrating an example of a functional configuration of the simulation computer according to the embodiment. FIG. 10 is a block diagram illustrating an example of a functional configuration of the simulation robot according to the embodiment. FIG. 11A is a sequence diagram illustrating an example of an operation flow of the simulation system according to the embodiment. FIG. 11B is a sequence diagram illustrating an example of an operation flow of the simulation system according to the embodiment.

Hereinafter, embodiments of the present invention will be described with reference to the drawings. Each of the embodiments described below shows a comprehensive or specific example. In addition, among the components in the following embodiments, components not described in the independent claims indicating the highest concept are described as arbitrary components. Each drawing in the accompanying drawings is a schematic diagram and is not necessarily illustrated exactly. Further, in each of the drawings, substantially the same components are denoted by the same reference numerals, and redundant description may be omitted or simplified in some cases. Further, in the present specification and claims, “apparatus” may mean not only one apparatus but also a system including a plurality of apparatuses.

<Simulation system configuration>
The configuration of the simulation system 1 according to the embodiment will be described. FIG. 1 is a diagram illustrating an example of a configuration of a simulation system 1 according to the embodiment. As shown in FIG. 1, the simulation system 1 includes one or more intermediary devices 10, one or more operation terminals 20, one or more simulation computers 40, and one or more simulation robots 50. including. The simulation computer 40 and the simulation robot 50 are examples of the simulation device 200. Hereinafter, when the “simulation computer 40” and the “simulation robot 50” are not distinguished, they may be simply referred to as “simulation device 200”. The mediation device 10, the operation terminal 20, the simulation computer 40, and the simulation robot 50 are connected via a communication network 90. Not all of the above components are not essential.

The communication network 90 is a network capable of performing data communication, for example, the Internet, an intranet, a wired or wireless LAN (Local Area Network), a WAN (Wide Area Network), a mobile communication network, a telephone line communication network, or another network. A communication network using wired or wireless communication may be used. In the present embodiment, the communication network 90 is the Internet.

The simulation computer 40 is a device that executes a simulation for causing a virtual robot model to perform a simulated operation. The virtual robot model is generated and operated on a computer program of the simulation computer 40. Note that the simulation computer 40 may or may not be a device specially generated for the simulation system 1, and may be, for example, an existing or general-purpose device used for simulating the operation of a robot. Good. The robot model of the simulation computer 40 may be a virtual robot model of a robot as exemplified by the actual robots 50A1 to 50An (n = 1,..., Natural numbers) of the simulation robot 50 described later. The virtual robot model may have a configuration including a robot main body as in the case of a simulated robot 50 to be described later, and may have a configuration including not only the robot main body but also an object to be processed and peripheral devices.

The simulation robot 50 is a device that executes a simulation for causing the actual robots 50A1 to 50An (n = 1,..., Natural numbers) to perform simulated operations as a robot model. The robots 50A1 to 50An may be any robot, for example, an industrial robot, a service robot, a construction machine, a tunnel excavator, a crane, a cargo handling vehicle, and a humanoid. The service robot is a robot used in various service businesses such as nursing, medical care, cleaning, security, guidance, rescue, cooking, and product supply. In the present embodiment, the robots 50A1 to 50An are industrial robots such as a vertical articulated robot, a horizontal articulated robot, a polar coordinate robot, a cylindrical coordinate robot, and a rectangular coordinate robot. Hereinafter, when the robots 50A1 to 50An are not distinguished, they may be simply referred to as “robot 50A”.

The simulation robot 50 includes any one of the robots 50A1 to 50An, and is provided for each of the robots 50A1 to 50An. The robots 50A1 to 50An, which are industrial robots, are provided for various tasks. Each of the robots 50A1 to 50An may be a robot actually arranged at a work site or a robot arranged for simulation. Each of the robots 50A1 to 50An may be configured to include, for example, a robot main body including a robot arm and an end effector. In addition to the robot main body, a processing target such as a work in a work, a transfer device, and an imaging device And a configuration including peripheral devices such as sensors. In the latter case, the simulation by the simulation robot 50 is a simulation in which all of the robot main body, the processing object, the peripheral devices, and the like are operated. Note that the simulation robot 50 may or may not be specially generated for the simulation system 1, and may be, for example, an existing or general-purpose robot used for simulating the operation of the robot. Good.

The operations performed by the robot models of the simulation computer 40 and the simulation robot 50 are operations including one or more individual operations such as horizontal movement, vertical movement, and rotation, and work. The above operation is a series of composite operations in which more individual operations than the above operations are combined in the order of execution. Examples of the above work are, for example, in the case of a robot model of an industrial robot, holding and moving an object with an end effector, holding the object with an end effector and assembling the object to be assembled, with an end effector Such operations include cutting an object and joining two or more objects with an end effector.

The operation terminal 20 is a device that executes a computer program for simulating the operation of the robot using the simulation device 200. The operation terminal 20 is connected to the simulation device 200 via the communication network 90, and can be operated by a user of the operation terminal 20. The user can execute the simulation by operating the operation terminal 20 even if the user is located remotely from the simulation device 200. The operation terminal 20 outputs information input by the user to the simulation device 200. Examples of the operation terminal 20 include a game machine, a game controller, a remote controller dedicated to a robot, a personal information terminal (PDA: Personal Data Assistant), a smart device such as a smartphone, a smart watch and a tablet, and a computer device such as a personal computer. And so on. Such an operation terminal 20 may or may not be a device specially generated for the simulation system 1, and may be an existing or general-purpose device.

In the present embodiment, the operation terminal 20 is a game device including a game machine and a game controller. Examples of the game device include a stationary game device and a portable game device. The operation terminal 20 is connected to the output device 30, and outputs an image signal and an audio signal for executing a simulation using the robot model to the output device 30, and the output device 30 outputs an image corresponding to the image signal and the audio signal. And voice to the user. For example, the output device 30 includes a display and a speaker. Examples of the display are a liquid crystal display (Liquid Crystal Display) and an organic or inorganic EL display (Electro-Luminescence Display).

The mediation device 10 mediates one or more operation terminals 20, one or more simulation computers 40, and one or more simulation robots 50 via a communication network 90. In the present embodiment, the mediation device 10 includes the information processing device 100. In the present embodiment, the mediation device 10 is a server, a computer device having a communication function and processing information.

The information processing device 100 is a device that manages a simulation of a robot operation using the operation terminal 20. The information processing apparatus 100 includes, for example, an arithmetic unit having a processor, a memory, and the like. The memory includes a semiconductor memory such as a volatile memory and a nonvolatile memory, and a storage device such as a hard disk (HDD: Hard Disc Drive) and an SSD (Solid State Drive). For example, the function of the arithmetic unit is a computer system (shown in the figure) including a processor such as a CPU (Central Processing Unit), a volatile memory such as a RAM (Random Access Memory), and a nonvolatile memory such as a ROM (Read-Only Memory). ). Part or all of the functions of the arithmetic unit may be realized by the CPU executing a program recorded in the ROM using the RAM as a work area. Note that some or all of the functions of the arithmetic unit may be realized by the computer system, or may be realized by a dedicated hardware circuit such as an electronic circuit or an integrated circuit, or may be realized by the computer system and the hardware circuit. It may be realized by a combination.

Specifically, the information processing device 100 may be configured by a computer device such as a computer and a personal computer, for example. Alternatively, the information processing apparatus 100 may include, for example, a microcontroller, an MPU (Micro Processing Unit), an LSI (Large Scale Integration), a system LSI, a PLC (Programmable Logic Controller), a logic circuit, and the like. Good. The plurality of functions of the information processing device 100 may be realized by being individually integrated into one chip, or may be realized by being integrated into one chip so as to include part or all. Further, each circuit may be a general-purpose circuit or a dedicated circuit. As an LSI, an FPGA (Field Programmable Gate Array) that can be programmed after manufacturing the LSI, a reconfigurable processor that can reconfigure the connection and / or setting of circuit cells inside the LSI, or a plurality of functions for specific applications An ASIC (Application \ Specific \ Integrated \ Circuit) in which the above circuits are integrated into one may be used.

The information processing apparatus 100 receives, from the operation terminal 20, first information that is information for operating the robot model input by the user to the operation terminal 20, and transmits the first information to the robot model of the simulation apparatus 200. Is converted into the second information which is information for operating. The information processing apparatus 100 transmits the second information to the simulation apparatus 200, and operates the second information according to the second information.

Further, the information processing apparatus 100 receives third information, which is information on the operation of the robot model, from the simulation apparatus 200 that operates according to the second information, and converts the third information into information that the operation terminal 20 can process. To the fourth information. For example, the information processing apparatus 100 receives third information indicating an individual operation of the robot model of the simulation apparatus 200, and converts the third information into fourth information as information functioning on an execution program of the operation terminal 20. Convert to By transmitting the fourth information to the operation terminal 20, the information processing apparatus 100 causes the operation terminal 20 to perform an operation of the robot model of the simulation device 200, or a robot model simulating the robot model or a corresponding model. , Through the output device 30 to the user. The corresponding model may be a model associated with the robot model, for example, it may not be similar in appearance to the robot model.

For example, the information processing apparatus 100 receives information of an image of the robot 50A of the simulated robot 50 captured by the imaging apparatus as third information, and transmits the information of the image to the operation terminal 20 as fourth information. Alternatively, the information may be presented to the operation terminal 20 as information on the operation of the robot model. Thus, the user can operate the simulation computer 40 or the robot model of the simulation robot 50 through the operation terminal 20 while visually recognizing the robot model.

Further, the information processing apparatus 100 is requested in the request information from a plurality of simulating apparatuses 200 based on the request information that is input by the user to the operation terminal 20 and is information of a simulation requesting execution. The simulation device 200 corresponding to the simulation is determined. Further, the information processing apparatus 100 connects the determined simulation apparatus 200 and the operation terminal 20 via the communication network 90, thereby enabling execution of a simulation using the same.

(4) The information processing device 100 evaluates the operation result of the robot model based on the information on the operation of the robot model received from the simulation device 200. The information processing apparatus 100 may compare the individual operation of the received robot model with the operation to be executed in the simulation, and determine the evaluation based on, for example, a difference between the two and a position and a required time. Good. Alternatively, the simulation device 200 can be configured to evaluate the result of the simulation. The information processing device 100 may receive the evaluation of the simulation result from the simulation device 200 and determine the evaluation set in the simulation system 1 based on the received evaluation. The information processing apparatus 100 registers the determined evaluation in the storage device as a user evaluation that is an evaluation for the user who operates the operation terminal 20. The storage device may be a storage device of the information processing device 100 or a storage device (not shown) included in the mediation device 10, and may be connected to the mediation device 10 via the communication network 90 or the like. The storage device of another device may be used. In addition, the information processing apparatus 100 transmits the determined evaluation to the operation terminal 20 so that the user is presented via the output device 30.

[Hardware configuration]
The hardware configuration of the operation terminal 20 will be described. FIG. 2 is a block diagram illustrating an example of a hardware configuration of the operation terminal 20 according to the embodiment. As shown in FIG. 2, the operation terminal 20 includes an operation terminal body 21 as a game machine and an operation device 22 as a game controller. The operation terminal main body 21 can execute a computer program for simulation, and the operation device 22 receives an input of an operation by a user. The operation terminal main body 21 includes a CPU 201, a ROM 202, a RAM 203, a memory 204, a media I / F (interface) 205, a network I / F 206, a first input / output I / F 207, an image processor 208, , A second input / output I / F 209, an audio processor 210, a third input / output I / F 211, and a recording medium drive 212 as constituent elements. Each component is connected by a bus or the like, but may be connected by any other wired or wireless communication. Not all of the above components are not essential.

The CPU 201 controls the operation of the operation terminal body 21 as a whole. The ROM 202 stores a program used for driving the CPU 201. The RAM 203 is used as a work area of the CPU 201. The memory 204 stores various data such as programs, and is configured by the storage device described above. The media I / F 205 controls reading or writing (storage) of data from or to a recording medium 60 such as a flash memory. For example, a computer program for executing a simulation may be stored in the recording medium 60 and executed by being read from the recording medium 60. Save data including a result of the simulation may be stored in the recording medium 60 or the memory 204. The network I / F 206 is an interface for performing data communication using the communication network 90. The first input / output I / F 207 is an interface that is connected to the operation device 22 and communicates signals and the like.

The image processor 208 includes a GPU (Graphics Processing Unit) that can generate a screen for performing a simulation. For example, the image processor 208 generates screen data such as an image of a robot model of the simulation computer 40 and the simulation robot 50, an image of a robot model simulating the robot model, and an image of a model corresponding to the robot model. Output to the two input / output I / F 209. The second input / output I / F 209 is an interface that is connected to the output device 30 and communicates screen data and image data.

The audio processor 210 includes a DSP (Digital Signal Processor) that generates audio for performing a simulation. The audio processor 210 amplifies the generated audio with an amplifier (not shown) including a D / A converter and outputs the amplified audio to the third input / output I / F 211. The third input / output I / F 211 is an interface that is connected to the output device 30 and communicates audio signals and the like. The recording medium drive 212 controls reading or writing of various types of data on the removable recording medium 70. For example, a computer program for executing a simulation may be stored in the recording medium 70 and executed by being read from the recording medium 70. Save data including a result of the simulation may be stored in the recording medium 70. Examples of the recording medium 70 are recording disks such as DVD-R, DVD-RW, BD (Blu-ray (registered trademark))-R, and BD-RE.

The operation device 22 accepts an input from the user. For example, the operation device 22 includes an input device 221 that is input via a user operation such as a cross button, a push switch, a dial, a joystick, a mouse, a key, and a touch panel. Further, the operation device 22 includes an operation input / output I / F 222 connected to the first input / output I / F 207 of the operation terminal main body 21 for communicating signals and the like. The input device 221 generates a signal indicating an input of the user, and outputs the signal to the first input / output I / F 207 via the operation input / output I / F 222. In the present embodiment, the configuration of the input device 221 is different from that of the robot dedicated operation device. For example, the number of input devices 221 and the method of inputting to the input devices 221 are different. The signal for the operation output from the dedicated operation device of the robot is different from the signal for the operation output from the input device 221 and is, for example, more complicated.

The hardware configuration of the mediation device 10 will be described. FIG. 3 is a block diagram illustrating an example of a hardware configuration of the mediation device 10 according to the embodiment. As shown in FIG. 3, the mediation device 10 includes a CPU 101, a ROM 102, a RAM 103, a memory 104, a network I / F 106, and an input / output I / F 107 as constituent elements. Each component is connected by a bus or the like, but may be connected by any other wired or wireless communication. Not all of the above components are not essential.

(4) The CPU 101 controls the operation of the entire intermediation device 10. The configurations and functions of the ROM 102, the RAM 103, and the memory 104 are the same as those of the ROM 202, the RAM 203, and the memory 204 of the operation terminal 20. The CPU 101, the ROM 102, the RAM 103, and the memory 104 constitute the information processing device 100. The configurations and functions of the network I / F 106 and the input / output I / F 107 are the same as those of the network I / F 206 and the first input / output I / F 207 of the operation terminal 20.

The input / output I / F 107 is an interface connected to the input device 113 for communicating signals and the like. The input device 113 receives an input by a user. For example, the input device 113 includes a device input via a user operation such as a button, a switch, a dial, a joystick, a mouse, a key, and a touch panel. The input device 113 generates a signal indicating a user input, and outputs the signal to the input / output I / F 107.

The hardware configuration of the simulation computer 40 will be described. FIG. 4 is a block diagram illustrating an example of a hardware configuration of the simulation computer 40 according to the embodiment. As shown in FIG. 4, the simulation computer 40 includes a CPU 401, a ROM 402, a RAM 403, a memory 404, a network I / F 406, a first input / output I / F 407, an image processor 408, and a second input / output An I / F 409 is included as a component. Each component is connected by a bus or the like, but may be connected by any other wired or wireless communication. Not all of the above components are not essential.

The CPU 401 controls the operation of the simulation computer 40 as a whole. The configurations and functions of the ROM 402, the RAM 403, and the memory 404 are the same as those of the ROM 202, the RAM 203, and the memory 204 of the operation terminal 20. The respective configurations and functions of the network I / F 406, the first input / output I / F 407, the image processor 408, and the second input / output I / F 409 are the same as the network I / F 206, the first input / output I / F 207, and the image processor of the operation terminal 20. 208 and the second input / output I / F 209.

The first input / output I / F 407 is an interface that is connected to the input device 413 and communicates signals and the like. The input device 413 receives an input by a user. The configuration of the input device 413 is the same as the input device 113 of the mediation device 10. Further, the image processor 408 generates screen data of an image of the robot model set in the simulation computer 40 and outputs the screen data to the second input / output I / F 409. The second input / output I / F 409 is an interface connected to the display 414 for communicating screen data and image data.

The hardware configuration of the simulation robot 50 will be described. FIG. 5 is a block diagram illustrating an example of a hardware configuration of the simulation robot 50 according to the embodiment. As shown in FIG. 5, the simulation robot 50 includes a control device 51, an imaging device 517, a display 514, an operation device 518, and a driving device 519. Not all of the above components are not essential.

The drive device 519 is a device that drives a movable portion of the robot 50A (see FIG. 1). For example, the robot 50A includes a robot arm and an end effector at the tip of the robot arm. The robot arm has a joint that bends and is movable at the joint. The end effector performs an operation of applying an action to a work such as gripping, suction, scooping, and hanging. The driving device 519 drives a joint of the robot arm, an end effector, and the like. Examples of the driving device 519 include an electric motor such as a servomotor, a vacuum generating device, and the like.

(4) The imaging device 517 captures an image of the robot 50A, and outputs captured image data to the control device 51. The imaging device 517 provides an image showing the state of the robot 50A to a user who performs a simulation in the simulation robot 50. In the present embodiment, the image is provided to the operation terminal 20. Examples of the imaging device 517 include a digital camera and a digital video camera.

The display 514 receives the image data captured by the imaging device 517 from the control device 51, and presents the image of the image data to the user. Further, the display 514 receives screen data for operation of the simulation robot 50 from the control device 51, and presents an image of the screen data to the user.

The operation device 518 receives an input of a command and information by a user who performs simulation in the simulation robot 50 and outputs a signal indicating the command and information to the control device 51. The operation device 518 includes devices input via user operations, such as a push switch, a lever, a dial, a joystick, a key, and a touch panel. In the present embodiment, operating device 518 is a dedicated operating device for robot 50A.

The control device 51 is a device that controls the entire simulation robot 50. The control device 51 includes a CPU 501, a ROM 502, a RAM 503, a memory 504, an imaging I / F 515, a network I / F 506, a first input / output I / F 507, an image processor 508, and a second input / output I / F. F509 and a drive circuit 516 are included as constituent elements. Each component is connected by a bus or the like, but may be connected by any other wired or wireless communication. Not all of the above components are not essential.

The CPU 501 controls the operation of the control device 51 as a whole. The configurations and functions of the ROM 502, the RAM 503, and the memory 504 are the same as those of the ROM 202, the RAM 203, and the memory 204 of the operation terminal 20. The configuration and function of each of the network I / F 506, the first input / output I / F 507, the image processor 508, and the second input / output I / F 509 are as follows. 208 and the second input / output I / F 209.

The first input / output I / F 507 is an interface that is connected to the operation device 518 and communicates signals and the like. The image processor 508 generates screen data to be displayed on the display 514 using the image data of the robot 50A captured by the imaging device 517 and outputs the screen data to the second input / output I / F 509. Further, the image processor 508 generates screen data for operating the simulation robot 50 and outputs the screen data to the second input / output I / F 509. The second input / output I / F 509 is an interface that is connected to the display 514 and communicates screen data and image data.

The imaging I / F 515 controls the driving of the imaging device 517 in response to the execution of the program. The imaging I / F 515 loads image data captured by the imaging device 517 into the RAM 503 or the memory 504 in response to the execution of the program. The imaging I / F 515 may include a circuit or the like for driving the imaging device 517.

The drive circuit 516 controls the drive of the drive device 519. For example, the driving circuit 516 supplies a current to each driving device 519 according to a command from the CPU 501, and controls driving of the driving device 519. The driving circuit 516 controls the driving of all the driving devices 519.

[Functional configuration]
The functional configuration of the operation terminal 20 will be described. FIG. 6 is a block diagram illustrating an example of a functional configuration of the operation terminal 20 according to the embodiment. As shown in FIG. 6, the operating device 22 includes an input unit 22a and an output unit 22b. The input unit 22a receives an input from a user, and its function is realized by the input device 221. The output unit 22b outputs information input to the input unit 22a to the operation terminal main body 21, and its function is realized by the operation input / output I / F 222. The information may be information such as ON or OFF of a button or the like of the input device 221, a duration of ON, and the like.

The operation terminal main body 21 includes a communication unit 21a, an input unit 21b, a program execution unit 21c, a reading unit 21d, a storage unit 21e, and an output unit 21f. Not all of the above components are not essential. The communication unit 21a communicates with the intermediary device 10 via the communication network 90, and its function is realized by the network I / F 206 or the like. The input unit 21b receives input of information from the operation device 22, and its function is realized by the first input / output I / F 207 and the like. The reading unit 21d reads information stored therein from the recording medium 60 and / or the recording medium 70, and the function is realized by the media I / F 205, the recording medium drive 212, and the like.

The storage unit 21e stores various information, and is realized by the ROM 202, the memory 204, the recording medium 60, the recording medium 70, and the like. For example, the storage unit 21e stores user information of each user. The user information may include identification information of the user such as a user ID, a simulation result of the user using the operation terminal 20, and the like. The simulation result may include a result of performing each simulation, an evaluation and an achievement, a proficiency of the user in each simulation, a rank or a rank of the user according to the proficiency, and the like. The rank and the rank may be absolute ranks corresponding to the proficiency, or may be relative ranks such as ranks by comparison with other users. The storage of the user information in the storage unit 21e may be automatically and periodically performed by the program execution unit 21c, or may be performed by the user performing an input or save operation using the operation device 22.

The output unit 21f outputs the image data and the audio data generated by the program execution unit 21c to the output device 30, and the function is realized by the second input / output I / F 209, the third input / output I / F 211, and the like.

The program execution unit 21c reads and executes a computer program stored in the recording medium 60, the recording medium 70, or the like. The computer program is a program for executing a simulation, and is a game program in the present embodiment. That is, in the present embodiment, the simulation is executed as a game on the operation terminal 20. Note that the program may be installed in the operation terminal 20 via the communication network 90, the recording medium 60, the recording medium 70, or the like as an application program, and may be stored in the ROM 202, the memory 204, or the like.

The program execution unit 21c operates according to a computer program using information received from the operation device 22 and information received via the communication network 90, and generates image data and audio data reflecting the received information. The program execution unit 21c outputs the generated image data and audio data to the output device 30 via the output unit 21f, and causes the user to present the data. In addition, the program execution unit 21c transmits operation information received from the operation device 22, user information stored in the storage unit 21e, and the like to the intermediary device 10 via the communication unit 21a and the communication network 90. The program execution unit 21c is realized by the CPU 201, the ROM 202, the RAM 203, the image processor 208, the audio processor 210, and the like.

The functional configuration of the mediation device 10 will be described. FIG. 7 is a block diagram illustrating an example of a functional configuration of the intermediary device 10 according to the embodiment. As shown in FIG. 7, the mediation device 10 includes a first communication unit 10a, a second communication unit 10b, a reception unit 10c, a determination unit 10d, a conversion unit 10e, an operation data processing unit 10f, a determination unit 10g and a storage unit 10h. Not all of the above components are not essential. The functions of the first communication unit 10a and the second communication unit 10b are realized by the network I / F 106 or the like. The function of the storage unit 10h is realized by the ROM 102, the memory 104, and the like. The functions of the receiving unit 10c, the determining unit 10d, the converting unit 10e, the operation data processing unit 10f, and the determining unit 10g are realized by the CPU 101, the ROM 102, the RAM 103, and the like.

The first communication unit 10a communicates with the operation terminal 20 via the communication network 90. The second communication unit 10b communicates with the simulation computer 40 or the simulation robot 50 via the communication network 90.

The storage unit 10h stores various information. For example, the storage unit 10h stores user information of each user in association with user identification information. Further, in the storage unit 10h, simulation permission conditions based on the rank or rank set by the user and the configuration of the simulation that can be executed under each permission condition are stored in association with each other. The configuration of the simulation includes the simulation content and the simulation device 200 that can execute the simulation content in association with each other. The simulation device 200 is a simulation computer 40 or a simulation robot 50. The simulation content includes the operation or operation of the robot model executed in the simulation, and may be set as, for example, the type of the simulation. In addition, the storage unit 10h may include an execution result of the simulation using the simulation device 200 and an evaluation related to the execution result in association with each other. In addition, the storage unit 10h may include the evaluation and the proficiency of the simulation corresponding to the evaluation in association with each other. Further, the storage unit 10h may include the simulation proficiency and the permission condition in association with each other. The storage unit 10h may include a threshold value or the like used by the determination unit 10g for determination such as evaluation.

The receiving unit 10c extracts the simulation contents executable by the user based on the user information received from the operation terminal 20, and presents the extracted simulation contents to the user. Specifically, the reception unit 10c receives the user information and the simulation execution command from the operation terminal 20, and extracts the permission condition corresponding to the user identification information included in the user information from the storage unit 10h. Further, the receiving unit 10c extracts a configuration of the simulation corresponding to the permission condition and transmits the extracted configuration to the operation terminal 20. The user can select from the configuration of the simulation presented by the operation terminal 20 to the output device 30.

For example, the operation terminal 20 causes the output device 30 to display a screen S as shown in FIG. FIG. 8 is a diagram illustrating an example of a simulation selection screen presented by the operation terminal 20 according to the embodiment. As shown in FIG. 8, the screen S displays any of the classes “A” to “D” as the user's permission condition for each operation or task, and further executes a simulation of each operation or task. The icon of the simulation device 200 that can perform the simulation is displayed. Therefore, by selecting the icon of the simulation device 200, the user can select the operation or operation for executing the simulation and the simulation device 200 used for the operation or operation. In FIG. 8, for example, the operation display “operation 1 @ class B” indicates that “operation 1” can be simulated by a user of “class B” or higher. For each operation display, the icon “simulated computer” indicates that the simulated computer 40 is available, and the icon “simulated robot A1 type” indicates that the simulated robot 50 including the robot 50A1 is available.

The determination unit 10d determines the configuration of the simulation to be executed by the user. The determination unit 10d receives from the operation terminal 20 the simulation content selected by the user on the operation terminal 20 and information on the simulation device 200. The determination unit 10d determines the received simulation content and the execution of the simulation in the simulation device 200, and transmits a command to execute the simulation content to the simulation device 200. The determination unit 10d connects the simulation device 200 and the operation terminal 20.

The conversion unit 10 e converts the first information for operating the robot model received from the operation terminal 20 into the second information for operating the robot model of the simulation device 200 during the execution of the simulation. The conversion unit 10e transmits the second information to the simulation device 200 that is executing the simulation. For example, when the simulation device 200 is the simulation computer 40, the second information is output from the input device 413 to the simulation computer 40 to cause the virtual robot model to execute an operation corresponding to the first information. Respond to information. When the simulation device 200 is the simulation robot 50, the second information corresponds to information output from the operation device 518 to the control device 51 to cause the robot 50A to perform an operation corresponding to the first information.

The operation data processing unit 10f converts the operation data of the robot model received from the simulation device 200 into operation data corresponding to the image display of the robot model on the operation terminal 20 during the execution of the simulation. 20.

For example, when the simulation device 200 is the simulation computer 40, the operation data received from the simulation device 200 may be composed of an image of the robot model, or may be composed of a simplified image showing the operation of the robot model. And numerical data such as a vector and coordinates representing the operation of the robot model. For example, the simplified image may be an image representing a skeleton of a robot model. The converted motion data may be composed of an image of a robot model displayed on the operation terminal 20 or may be composed of an image of a virtual robot model generated by the simulation computer 40. It may be composed of a simplified image showing the operation, or may be numerical data representing the operation of the robot model.

For example, when the simulation device 200 is the simulation robot 50, the operation data received from the simulation device 200 may be configured by image data of the robot 50A captured by the imaging device 517. The converted operation data may be composed of captured image data of the robot 50A, or may be composed of a simplified image showing the operation of the robot 50A by a characteristic portion or the like in the captured image. May be numerical data representing the operation of. The simplified image may be an image of a virtual robot model, and in this case, may be an image using CG (Computer Graphics) or animation. The virtual robot model may be similar or dissimilar to the robot 50A.

The determination unit 10g evaluates the result of the simulation executed by the simulation device 200. Furthermore, the determination unit 10g determines the user's proficiency based on the evaluation, and further determines the user's rank or rank based on the proficiency. The determination unit 10g transmits at least one piece of information of evaluation, proficiency, rank, and class to the operation terminal 20, and the operation terminal 20 causes the output device 30 to present the received information. In addition, the determination unit 10g causes the storage unit 10h to store the evaluation, the proficiency level, the rank, and the class as new user information.

The determination unit 10g may determine the evaluation based on the difference between the operation of the robot model detected by the operation data processing unit 10f and the operation of the robot model to be executed in the simulation and the threshold value. . The difference may include a difference in the position of each part of the robot model, a difference in time required for the operation, and the like. Alternatively, the simulation device 200 may evaluate the result of the simulation and transmit the result to the determination unit 10g, and the determination unit 10g may use the evaluation.

The functional configuration of the simulation computer 40 will be described. FIG. 9 is a block diagram illustrating an example of a functional configuration of the simulation computer 40 according to the embodiment. As shown in FIG. 9, the simulation computer 40 includes a communication unit 40a, a reception unit 40b, a program execution unit 40c, a determination unit 40d, an operation data output unit 40e, and a storage unit 40f. Not all of the above components are not essential. The function of the communication unit 40a is realized by the network I / F 406 and the like. The function of the storage unit 40f is realized by the ROM 402, the memory 404, and the like. The functions of the reception unit 40b, the program execution unit 40c, the determination unit 40d, and the operation data output unit 40e are realized by the CPU 401, the ROM 402, the RAM 403, and the like.

The communication unit 40a communicates with the mediation device 10 via the communication network 90. The storage unit 40f stores various information. For example, the storage unit 40f may store, for each simulation content executable by the simulation computer 40, information on the position and required time of each operation of the robot model executed in the simulation. The storage unit 40f may store a threshold value or the like used by the determination unit 40d for evaluation.

The receiving unit 40b receives, from the intermediary device 10, a command to execute a simulation and the content of the simulation to be executed. The receiving unit 40b outputs the received instruction to execute the simulation content to the program executing unit 40c. After the start of the simulation, the reception unit 40b receives the second information for operating the virtual robot model from the conversion unit 10e of the intermediation device 10, and outputs the second information to the program execution unit 40c.

(4) The program execution unit 40c executes the computer program of the instructed simulation contents. The program is stored in the ROM 402 or the memory 404. The program execution unit 40c causes the virtual robot model to execute an operation corresponding to the second information according to the second information received from the reception unit 40b and the computer program.

The operation data output unit 40e detects operation result data indicating the operation result from the robot model operated by the program execution unit 40c. The operation result data may include an image of the robot model, a simplified image of the skeleton of the robot model, and the like, the position of each part of the robot model, the time required for the position, and the like. Further, the operation data output unit 40 e generates operation data from the operation result data, and transmits the operation data to the intermediary device 10. The operation data is the operation data described above in the description of the operation data processing unit 10f of the mediation device 10.

The determination unit 40d evaluates the result of the simulation using the operation data generated by the operation data output unit 40e, and transmits the evaluation to the intermediary device 10. The determination unit 40d determines the evaluation, similarly to the determination unit 10g of the mediation device 10. Furthermore, the determination unit 40d may determine the user's proficiency level, rank, rank, and the like, and transmit the determined level to the intermediary device 10.

The functional configuration of the simulation robot 50 will be described. FIG. 10 is a block diagram illustrating an example of a functional configuration of the simulation robot 50 according to the embodiment. As shown in FIG. 10, the simulation robot 50 includes a communication unit 50a, a reception unit 50b, an imaging control unit 50c, an image processing unit 50d, a drive control unit 50e, a state detection unit 50f, and a determination unit 50g. , A storage unit 50h, an imaging device 517, a driving device 519, and the like. Not all of the above components are not essential. The communication unit 50a, the reception unit 50b, the imaging control unit 50c, the image processing unit 50d, the drive control unit 50e, the state detection unit 50f, the determination unit 50g, and the storage unit 50h constitute the control device 51. The function of the communication unit 50a is realized by the network I / F 506 and the like. The function of the storage unit 50h is realized by the ROM 502, the memory 504, and the like. The functions of the reception unit 50b, the imaging control unit 50c, the image processing unit 50d, the drive control unit 50e, the state detection unit 50f, and the determination unit 50g are realized by the CPU 501, the ROM 502, the RAM 503, and the like.

The communication unit 50a communicates with the mediation device 10 via the communication network 90. The storage unit 50h stores various information. For example, the storage unit 50h may store, for each simulation content that can be executed by the simulated robot 50, information on the position, acting force, and required time of each operation of the robot 50A executed in the simulation. An example of the acting force is a force that pushes or pulls the object by the end effector of the robot. In addition, the storage unit 50h may store a threshold value or the like used by the determination unit 50g for evaluation.

The receiving unit 50b receives, from the intermediation device 10, a command to execute a simulation and the content of the simulation to be executed. The receiving unit 50b outputs the received instruction to execute the simulation content to the imaging control unit 50c and the drive control unit 50e. After the start of the simulation, the reception unit 50b receives second information for operating the robot 50A from the conversion unit 10e of the intermediation device 10, and outputs the second information to the drive control unit 50e.

When the imaging control unit 50c receives the instruction to execute the simulation, the imaging control unit 50c activates the imaging device 517 to perform imaging. The imaging device 517 captures an image of the robot 50A, for example, its end effector, and outputs the captured image to the image processing unit 50d. The image processing unit 50d transmits the image data of the image received from the imaging device 517 to the mediation device 10.

The drive control unit 50e drives the driving device 519 according to the second information received from the reception unit 50b, thereby causing the robot 50A to execute an operation corresponding to the second information.

(4) The state detection unit 50f detects the state of each unit of the robot 50A, and outputs a detection result to the determination unit 50g. For example, the state detection unit 50f may detect the three-dimensional position of each unit by detecting the amount of movement of the movable part of the robot 50A as the above state. At this time, the state detection unit 50f may detect the operation amount of the movable part of the robot 50A from the detection value of the rotation sensor provided in the driving device 519. In addition, the state detection unit 50f may detect, as the state, an acting force acting on the end effector of the robot 50A. At this time, the state detection unit 50f may detect the acting force from a detection value of a force sensor such as a force sensor provided in the end effector. The state detection unit 50f may include a timer, and detect, as the state, the time required between the operations of the robot 50A. The drive control unit 50e may control the drive of the drive device 519 using the detection result of the state detection unit 50f as feedback information.

The determination unit 50g evaluates the result of the simulation using the detection result of the state detection unit 50f, and transmits the evaluation to the intermediary device 10. The determination unit 50g determines the evaluation in the same manner as the determination unit 10g of the mediation device 10. Furthermore, the determination unit 50g may determine the user's proficiency, rank, rank, and the like, and transmit the determined result to the mediation device 10.

<Operation of the simulation system>
The operation of the simulation system 1 according to the embodiment will be described. FIGS. 11A and 11B are sequence diagrams illustrating an example of an operation flow of the simulation system 1 according to the embodiment. As shown in FIGS. 11A and 11B, the user starts a simulation game program on the operation terminal 20 (step S101). Next, the user inputs user identification information and the like, which are user information, to the operation terminal 20, and the operation terminal 20 transmits the identification information and the like to the mediation device 10 (step S102). That is, the user logs in to the simulation game. The user identification information is registered in the mediation device 10 in advance.

Next, the mediation device 10 authenticates the received user identification information, extracts a simulation permission condition corresponding to the identification information, and extracts a simulation configuration corresponding to the extracted permission condition. Then, the mediation apparatus 10 transmits the extracted simulation configuration to the operation terminal 20 as a simulation configuration executable by the user (step S103). The configuration of the simulation is a combination of the simulation contents and the simulation device 200 that executes the simulation contents.

Next, using the operation terminal 20, the user determines the configuration of the simulation to be executed from the configuration of the simulation, which is a combination of the executable simulation contents displayed on the output device 30 and the simulation device 200. The operation terminal 20 transmits the determined configuration of the simulation to the mediation device 10 (Step S104).

Next, the mediation device 10 determines whether or not the simulation device 200 included in the received simulation configuration is the simulation computer 40 (step S105). If the mediation device 10 is the simulated computer 40 (Yes in step S105), the process proceeds to step S106. If the mediation device 10 is not the simulated computer 40 (No in step S105), the process proceeds to step S107.

In step S106, the mediation apparatus 10 extracts the simulation computer 40 capable of executing the simulation contents included in the simulation configuration from the simulation computers 40 to which the mediation apparatus 10 can be connected, and performs the simulation. The computer 40 and the operation terminal 20 are connected. Further, the mediation device 10 transmits a command to execute the simulation contents to the simulation computer 40.

Next, the simulation computer 40 executes the instructed simulation contents (step S108). The simulation computer 40 transmits a notification indicating the execution of the simulation to the operation terminal 20 via the mediation device 10.

During the execution of the simulation, the operation terminal 20 transmits the first information indicating the operation input by the user to the mediation device 10 (Step S109). Further, the mediation device 10 converts the first information into the second information for operating the virtual robot model of the simulation computer 40, and transmits the second information to the simulation computer 40 (step S110). The simulation computer 40 operates the robot model according to the second information, and transmits operation data of the robot model to the mediation device 10 (step S111). The intermediary device 10 converts the received operation data into operation data corresponding to presentation of a robot model image display or the like on the operation terminal 20, and transmits the operation data to the operation terminal 20 (step S112). The operation terminal 20 causes the output device 30 to present an image or the like of a robot model that executes an operation corresponding to the received operation data. Note that the simulation computer 40 may transmit image data of a virtual robot model to the intermediary device 10 as motion data, and the intermediary device 10 may transmit the received image data to the operation terminal 20. The operation terminal 20 may cause the output device 30 to display the received image data.

Next, the operation terminal 20 determines whether or not all the operations included in the simulation being executed have been completed (step S113). If the operation has been completed (Yes in step S113), the operation terminal 20 proceeds to step S114, transmits a notification of completion to the intermediary device 10, and if not completed (No in step S113), returns to step S109.

Next, in step S115, the mediation apparatus 10 notifies the simulation computer 40 of the completion of all the operations included in the simulation. Next, the simulation computer 40 evaluates the result of the simulation based on the operation data of the robot model through the simulation, and transmits the evaluation result to the intermediary device 10 (step S116).

Next, in step S117, the intermediary device 10 determines the evaluation of the simulation result, the user's proficiency, the rank, the rank, and the like based on the received evaluation result, and transmits at least one of these to the operation terminal 20. . Further, the mediation device 10 registers the evaluation result, the proficiency, the rank, the class, and the like as new user information in itself. Next, the operation terminal 20 outputs the received evaluation result, proficiency level, rank, rank, and the like to the output device 30 to be presented to the user (step S118). When the user stops the operation terminal 20, the operation terminal 20 ends the simulation game program (step S119).

In step S107, the mediation device 10 includes a robot 50A capable of executing the simulation contents included in the configuration of the simulation determined in step S104 from the simulated robots 50 to which the mediation device 10 can be connected. The simulation robot 50 is extracted, and the simulation robot 50 and the operation terminal 20 are connected. Further, the mediation device 10 transmits a command to execute the simulation contents to the simulation robot 50.

Next, the simulating robot 50 executes the instructed simulation contents (step S120). Next, the control device 51 of the simulation robot 50 causes the imaging device 517 to execute imaging of the robot 50A (step S121). The simulation robot 50 transmits a notification indicating the execution of the simulation to the operation terminal 20 via the intermediary device 10.

During the execution of the simulation, the operation terminal 20 transmits the first information to the mediation device 10 (step S122), and the mediation device 10 converts the first information into the second information for operating the robot 50A, and transmits the second information to the simulation robot 50. It transmits (step S123). The control device 51 operates the robot 50A according to the second information (step S124). At this time, the control device 51 detects the state of each part of the robot 50A. Next, the control device 51 transmits the image data captured by the imaging device 517 to the mediation device 10 (step S125). The mediation device 10 transmits the received image data to the operation terminal 20 (Step S126). The operation terminal 20 causes the output device 30 to present an image or the like of the received image data.

Next, the operation terminal 20 determines whether or not all the operations included in the simulation being executed have been completed (Step S127). If the operation has been completed (Yes in step S127), the operation terminal 20 proceeds to step S128, transmits a notification of completion to the intermediary device 10, and if not completed (No in step S127), returns to step S122.

Next, in step S129, the mediation apparatus 10 notifies the simulation robot 50 of the completion of all the operations included in the simulation. Next, the control device 51 of the simulation robot 50 evaluates the result of the simulation based on the state of each unit of the robot 50A through the simulation, and transmits the evaluation result to the intermediary device 10 (step S130).

Next, in step S131, the intermediary device 10 determines the evaluation of the simulation result, the proficiency of the user, the rank, the rank, and the like based on the received evaluation result, and transmits at least one of these to the operation terminal 20. And register for itself. Next, the operation terminal 20 outputs the received evaluation result and the like to the output device 30 and presents it to the user (step S132). When the user stops the operation terminal 20, the operation terminal 20 ends the simulation game program (step S133).

By the processes in steps S101 to S133 described above, the user located at a remote place from the simulation apparatus 200 can use the operation terminal 20 owned by the user to execute the simulation computer 40 or the simulation robot via the communication network 90. A simulation using 50 can be performed. As described above, the user can improve the skill level of the operation of the robot by executing the simulation using the operation terminal 20. For example, in an environment where the operation terminal 20 and the robot of the actual machine are connected via the communication network 90 and the robot can be operated by the operation terminal 20, the user can use the operation terminal 20 to operate the actual machine while being at a remote place. Will be able to do the actual work with that robot.

<Effects>
In the simulation system 1 according to the above-described embodiment, the information processing device 100 of the intermediary device 10 is connected to the simulation device 200 that executes a simulation for causing the robot model to perform a simulated operation, and is connected to the simulation device 200. Converts the first information for operating the robot model, which is input to the operation terminal 20 operated by a remote user, into second information for operating the robot model of the simulation device 200, And causes the operation terminal 20 to present information on the operation of the robot model of the simulation device 200 that operates according to the second information. The operation terminal 20 and the simulation device 200 may be connected via a communication network 90. Further, for example, the simulated operation of the robot model may be an operation simulating the operation performed by the robot of the real machine.

According to the above configuration, the information processing apparatus 100 converts the first information corresponding to the operation input to the operation terminal 20 into the second information, and causes the simulation apparatus 200 to operate according to the second information, thereby performing the simulation. The robot model of the device 200 performs an operation corresponding to the above operation. Therefore, the user can simulate the operation of the robot using the operation terminal 20 at a location remote from the simulation device 200. Furthermore, since information transmitted and received between the operation terminal 20 and the simulation device 200 is converted into information in a format adapted to each, the user may use, for example, the operation terminal 20 that the user is accustomed to. it can. Therefore, the user can easily execute the simulation operation, and easily accept the simulation. Further, by connecting the operation terminal 20 and the simulation device 200 via the communication network 90, various operation terminals 20 can be connected to the simulation device 200. Therefore, various users can access the simulation device 200 via the operation terminal 20 and simulate the operation of the robot, and thereby can operate the robot.

In the simulation system 1 according to the embodiment, the operation terminal 20 may be at least one of a game machine, a game controller, a remote controller dedicated to a robot, a personal information terminal, a smart device, and a personal computer. According to the above configuration, simulation using various operation terminals 20 can be performed.

Further, the information processing apparatus 100 according to the embodiment, based on the request information which is input to the operation terminal 20 and is information of a simulation requesting execution, requests a request from the plurality of simulation apparatuses 200 in the request information. The simulation device 200 corresponding to the simulation to be performed may be determined, and the simulation using the determined simulation device 200 and the operation terminal 20 may be performed. According to the above configuration, the information processing apparatus 100 can cause the user to execute various simulations in response to a request from the user.

Further, the information processing apparatus 100 according to the embodiment evaluates the operation result of the robot model based on the information on the operation of the robot model of the simulation apparatus 200, and performs the evaluation as a user evaluation that is an evaluation for a user who operates the operation terminal 20. Alternatively, the evaluation of the operation result may be registered. Further, the information processing apparatus 100 determines a simulation and a simulation apparatus 200 that can be executed by the user to be evaluated by the user based on the registered user evaluation, and transmits information of the determined simulation and the simulation apparatus 200 to an operation terminal. 20 may be presented.

According to the above configuration, the information processing apparatus 100 registers the evaluation of the simulation result as the user evaluation of the operation terminal 20. As a result, the registered user evaluation can be used. For example, the information processing apparatus 100 can present the registered user evaluation to the user when performing a simulation or the like. Thus, the user can easily determine the configuration of the simulation to be executed according to his / her own user evaluation. Further, when the user operates the actual robot, it is possible to determine the robot and the work content suitable for the user based on the user evaluation. In addition, the user or the provider of the simulation (for example, the mediation apparatus 10) can determine the simulation content and the simulation apparatus 200 that are suitable or executable for the user.

In addition, the information processing apparatus 100 according to the embodiment may convert information on the operation of the robot model of the simulation apparatus 200 into information that functions on an execution program of the operation terminal 20. According to the above configuration, the information processing apparatus 100 can associate the operation of the robot model in the simulation device 200 with the operation on the program executed on the operation terminal 20.

Further, the information processing apparatus 100 according to the embodiment enables execution of a simulation using the simulation computer 40 and the operation terminal 20 that cause the virtual robot model to perform a simulated operation, and the simulation computer 40 generates the simulation. The operation terminal 20 may present information on the operation of the virtual robot model to be performed. According to the above configuration, the user can execute the simulation of the simulation computer 40 using the operation terminal 20.

Further, the information processing apparatus 100 according to the embodiment enables execution of a simulation using the simulated robot 50 that causes the actual robot 50 </ b> A to perform a simulated operation as a robot model and the operation terminal 20. Information on the image of the robot 50A to be imaged may be presented to the operation terminal 20 as information on the operation of the robot model. According to the above configuration, the user can execute a simulation using the actual robot 50 </ b> A using the operation terminal 20. Further, the user can execute the simulation while visually recognizing the image of the actual robot 50A.

Further, the information processing apparatus 100 according to the embodiment converts an image of the robot 50A of the real machine into an image of a virtual robot model, and transmits information on the operation of the virtual robot model in the converted image to the operation terminal 20. It may be presented. For example, the image of the virtual robot model may be an image using CG, animation, or the like. According to the above configuration, the image presented to the user by the operation terminal 20 can be an image that is familiar to the user. Therefore, the user becomes familiar with the simulation.

The mediation device 10 according to the embodiment includes the information processing device 100 according to the embodiment, and mediates the operation terminal 20 and the simulation device 200 via the communication network 90. According to the above configuration, the same effects as those of the information processing apparatus 100 according to the embodiment can be obtained.

The simulation system 1 according to the embodiment includes an information processing apparatus 100 according to the embodiment, a simulation apparatus 200, and an intermediary apparatus that mediates the operation terminal 20 and the simulation apparatus 200 via a communication network 90. 10 is provided. According to the above configuration, the same effects as those of the information processing apparatus 100 according to the embodiment can be obtained. Note that the simulation system 1 may further include an operation terminal 20.

<Other embodiments>
Although the embodiments of the present invention have been described above, the present invention is not limited to the above embodiments. That is, various modifications and improvements are possible within the scope of the present invention. For example, various modifications made to the embodiments, and forms constructed by combining components in different embodiments are also included in the scope of the present invention.

For example, in the embodiment, the information processing device 100 is mounted on the mediation device 10, but is not limited thereto. For example, the information processing device 100 may be mounted on the operation terminal 20 or the simulation device 200. Further, the information processing device 100 may be divided and mounted on two or more devices. For example, a first information processing device and a second information processing device having some functions of the information processing device 100 are respectively mounted on two of the mediation device 10, the operation terminal 20, and the simulation device 200. Is also good. Alternatively, the first to third information processing apparatuses having some functions of the information processing apparatus 100 may be mounted on the mediation apparatus 10, the operation terminal 20, and the simulation apparatus 200, respectively.

The simulation system 1 according to the embodiment includes the operation terminal 20, the mediation device 10, and the simulation device 200, but is not limited thereto. For example, the simulation system 1 does not need to include the mediation device 10.

In the simulation system 1 according to the embodiment, the simulation computer 40 and the simulation robot 50 are configured to operate the robot model according to a preset computer program and data. Not done. For example, the simulation computer 40 and the simulation robot 50 may be configured to use a digital twin environment. In this case, the simulation computer 40 and the simulation robot 50 may be configured to operate using, for example, various data accumulated on the cloud, that is, big data. For example, data is data obtained by various users executing various simulations using various simulated computers 40 and simulated robots 50 and the like, and actually performing various operations using various robots. May be included.

The present invention may be an information processing method. For example, the information processing method according to one embodiment of the present invention is connected to a simulation device that executes a simulation that causes a robot model to perform a simulated operation, and is input to an operation terminal operated by a remote user of the simulation device. The converted first information for operating the robot model is converted to second information for operating the robot model of the simulation device, and operating the simulation device according to the second information, Information on the operation of the robot model of the simulation device that operates according to the two information is presented to the operation terminal. According to this method, the same effects as those of the information processing apparatus 100 and the like can be obtained. Such an information processing method may be realized by a circuit such as a CPU or an LSI, an IC card, a single module, or the like.

The present invention may be a program for executing the above information processing method, or may be a non-temporary computer readable recording medium on which the above program is recorded. Needless to say, the program can be distributed via a transmission medium such as the Internet.

数字 Further, the numbers such as ordinal numbers and quantities used above are all examples for specifically explaining the technology of the present invention, and the present invention is not limited to the illustrated numbers. Further, the connection relation between the constituent elements is illustrated for specifically explaining the technology of the present invention, and the connection relation for realizing the function of the present invention is not limited to this.

The division of blocks in the functional block diagram is merely an example, and a plurality of blocks may be implemented as one block, one block may be divided into a plurality of blocks, and / or some functions may be moved to another block. Good. In addition, a single piece of hardware or software may process functions of a plurality of blocks having similar functions in parallel or in a time-division manner.

1 simulation system 10 intermediary device 20 operation terminal 40 simulation computer (simulation device)
50 simulation robot (simulation device)
50A, 50A1 to 50An Robot 90 Communication network 100 Information processing device 200 Simulating device 517 Imaging device

Claims

First information for operating the robot model, which is connected to a simulation device that executes a simulation that causes the robot model to perform a simulated operation and is input to an operation terminal operated by a remote user of the simulation device, Converting to the second information for operating the robot model of the simulation device,
Operating the simulation device according to the second information,
An information processing device that causes the operation terminal to present information on the operation of the robot model of the simulation device that operates according to the second information.
Based on request information that is input to the operation terminal and is information on a simulation requesting execution, the simulation device corresponding to the simulation requested in the request information is determined from a plurality of the simulation devices. And
The information processing device according to claim 1, wherein a simulation using the determined simulation device and the operation terminal is enabled.
Based on the information on the operation of the robot model of the simulation device, evaluate the operation result of the robot model,
The information processing device according to claim 1, wherein an evaluation of the operation result is registered as a user evaluation that is an evaluation for a user who operates the operation terminal.
Based on the registered user evaluation, determine a simulation and the simulation device that can be executed by the user of the user evaluation target,
The information processing device according to claim 3, wherein the determined information of the simulation and the simulation device is presented to the operation terminal.
The information processing apparatus according to any one of claims 1 to 4, wherein the simulated operation of the robot model is an operation simulating an operation performed by a robot of a real machine.
Enables execution of simulation using the simulation device and the operation terminal as a computer device that causes the virtual robot model to perform a simulated operation,
The information processing device according to any one of claims 1 to 5, wherein information on an operation of the virtual robot model generated by the simulation device is presented to the operation terminal.
As the robot model, it is possible to execute a simulation using the simulating device and the operation terminal for performing a simulated operation of a real robot,
The information processing apparatus according to any one of claims 1 to 6, wherein information on an image of the robot imaged by an imaging apparatus is presented to the operation terminal as information on an operation of the robot model.
Convert the image of the robot to a virtual robot model image,
The information processing device according to claim 7, wherein information on an operation of the virtual robot model in the converted image is presented to the operation terminal.
The information processing device according to any one of claims 1 to 8, wherein the operation terminal and the simulation device are connected via a communication network.
The operation terminal according to any one of claims 1 to 9, wherein the operation terminal is at least one of a game machine, a game controller, a remote controller dedicated to a robot, a personal information terminal (PDA), a smart device, and a personal computer. An information processing apparatus according to claim 1.
An information processing apparatus according to any one of claims 1 to 10, further comprising:
A mediation device that mediates the operation terminal and the simulation device via a communication network.
An information processing apparatus according to any one of claims 1 to 10,
The simulating device;
A simulation system including an intermediary device that mediates the operation terminal and the simulation device via a communication network.
The simulation system according to claim 12, further comprising the operation terminal.
First information for operating the robot model, which is connected to a simulation device that executes a simulation that causes the robot model to perform a simulated operation and is input to an operation terminal operated by a remote user of the simulation device, Converting to the second information for operating the robot model of the simulation device,
Operating the simulation device according to the second information,
An information processing method for causing the operation terminal to present information on the operation of the robot model of the simulation device that operates according to the second information.
Based on request information that is input to the operation terminal and is information on a simulation requesting execution, the simulation device corresponding to the simulation requested in the request information is determined from a plurality of the simulation devices. And
The information processing method according to claim 14, wherein a simulation using the determined simulation device and the operation terminal is enabled.
Based on the information on the operation of the robot model of the simulation device, evaluate the operation result of the robot model,
The information processing method according to claim 14, wherein an evaluation of the operation result is registered as a user evaluation that is an evaluation for a user who operates the operation terminal.
Based on the registered user evaluation, determine a simulation and the simulation device that can be executed by the user of the user evaluation target,
The information processing method according to claim 16, wherein the determined information of the simulation and the simulation device is presented to the operation terminal.
The information processing method according to any one of claims 14 to 17, wherein the simulated operation of the robot model is an operation simulating an operation performed by a robot of a real machine.
Enables execution of simulation using the simulation device and the operation terminal as a computer device that causes the virtual robot model to perform a simulated operation,
19. The information processing method according to claim 14, wherein information on an operation of the virtual robot model generated by the simulation device is presented to the operation terminal.
As the robot model, it is possible to execute a simulation using the simulating device and the operation terminal for performing a simulated operation of a real robot,
The information processing method according to any one of claims 14 to 19, wherein information on an image of the robot captured by an imaging device is presented to the operation terminal as information on an operation of the robot model.
Convert the image of the robot to a virtual robot model image,
The information processing method according to claim 20, wherein information on an operation of the virtual robot model in the converted image is presented to the operation terminal.
22. The information processing method according to claim 14, wherein the operation terminal and the simulation device are connected via a communication network.
The information according to any one of claims 14 to 22, wherein the operation terminal is at least one of a game machine, a game controller, a remote controller dedicated to a robot, a personal digital assistant (PDA), a smart device, and a personal computer. Processing method.