KR102393627B1 - Mr environment and robot interaction training system - Google Patents

Abstract

The present invention relates to a training system for interaction between an MR environment and a robot, wherein a robot including a camera and a sensor is driven in a training space, and a virtual environment such as a real environment is provided to the trainee to train the trainee .

Description

MR ENVIRONMENT AND ROBOT INTERACTION TRAINING SYSTEM

The present invention establishes a disaster scenario of the MR environment and trains the robot to drive the robot. Interaction training between the MR environment and the robot to achieve the training goal by coping with various events by building the environment in which the trainee controls the robot as the MR environment It's about the system.

The era of the 4th industrial revolution demands the speed of rapid change in the modern society that pursues technological innovation, and robots and artificial intelligence have become one of the fields of attention in the modern society.

These robots have been developed for the purpose of replacing many of the tasks of humans, and in the near future, it is expected that they will take over the tasks of rescuing people at disaster sites and restoring damaged facilities.

Disasters, such as fire, flood, and landslide, require very important responses and situational judgment at every moment.

In a disaster situation, flying robots such as ground-running robots or drones, or various types of robots are developed and operated as needed.

In the event of a disaster, the controller's ability to control and respond appropriately to the disaster response robot is required.

The existing evaluation system for training and evaluating the disaster response capability of the controller is mainly limited to training and evaluating the operational capability of the controller in a virtual environment.

There is a need for a system that trains or evaluates the ability to handle a variety of disaster situations while operating real robots and issuing timely commands.

The key element of this system for training and evaluation of disaster situations is to make the trainees who control the real robot feel the virtual environment the same as the real one, but it is necessary to make the robot in the real environment recognize the disaster mission in the virtual environment. Since there is no system to control it, it has become a limitation of the virtual environment.

For example, in the real virtual simulation training system called LVC (Live-Virtual-Constructive), since training in tactics and recognition of virtual enemies are the main points, an environment in which the real control robot and the virtual medium are in contact is not created. Therefore, there is a difference from the disaster environment, and in addition, many training systems having a virtual simulation training system do not use real robots, so there is a problem that trainees feel a sense of heterogeneity in the real situation.

The present invention is an invention devised to solve the problems of the prior art, and has an object to provide a system in which trainees train through a robot to respond to a disaster in an MR environment.

In particular, the present invention copes with unexpected situations in a scenario by visually confirming the degree of damage to the virtual training event and the robot in contact with it and the virtual training event so that the trainee feels that the robot in the real environment recognizes the disaster mission in the virtual environment. The purpose of training is to provide a system for training to properly operate a robot in a disaster situation.

The problems of the present invention are not limited to the problems mentioned above, and other problems not mentioned will be clearly understood by those skilled in the art from the following description.

The MR environment and robot interaction training system of the present invention for achieving the above object is a training system for training a trainee to respond to a disaster mission by manipulating a robot in a disaster situation, wherein the trainee By manipulating the robot, environmental factors that may occur during the disaster mission are set as a training event, and the trainee drives the robot to perform the disaster mission and training in response to the training event. A scenario module to build, a mixed environment realization module for realizing the disaster mission in virtual or real form in the scenario, determining whether the training event is real or virtual in the scenario, and the trainee using the robot to perform the disaster mission When the virtual training event occurs during operation, the robot restricts the operation of the robot due to the virtual training event before the trainee commands the robot to control the robot so that the trainee recognizes the virtual training event When the operation of the robot is restricted in the interaction module, the robot driving module and the robot are sent to the robot by giving priority to the command to limit the driving of the robot rather than the command by the trainee to control the robot and the robot is the disaster and a control module providing the trainee with a result driven by the trainee's command while performing a task and a result of limiting the operation of the robot by the interaction module, wherein the interaction module is configured to drive the robot in the virtual training event A change in the robot and a change in the virtual training event in response to a result of limiting may be provided to the control module.

And the interaction module determines whether the training event is an actual training event or a virtual training event in the scenario, and when the virtual training event is the virtual training event, the result of the trainee commanding the robot to drive the robot corresponds to the virtual training event an interaction determination unit that determines whether the result of the command to Through the interaction control unit provided to the robot driving module and the interaction control unit, a signal stimulating the senses of the trainee is provided to the control module to indicate the state of the robot whose operation is restricted by the virtual training event to the control module so that the trainee is the robot. It may include an interaction effect unit for recognizing a change in the robot and a change in the virtual training event with respect to the result of limiting the driving.

In addition, in the interaction module, when the interaction determining unit determines that the training event is the actual training event, the interaction control unit may not be involved in driving the robot.

In addition, the interaction control unit may extract the training event information from the scenario module and generate a control command that is a command for limiting the operation of the robot in response to the size and location of the virtual training event.

In addition, the interaction effect unit analyzes the command that the trainee manipulated the robot, and the robot is deformed and the virtual training event is deformed according to the speed of the robot and the number of times the robot contacts and collides with the virtual training event. It is possible to express the obtained state as the mixed environment.

In addition, the robot may include a robot action unit that physically changes the robot state of the robot so that the trainee can recognize a result corresponding to the virtual training event.

In addition, the robot is provided with wheels to be movable, and the robot action unit is mounted on the wheel of the robot to block the driving force to rotate the wheel by the control command.

The trainee may further include a training evaluation module for evaluating a result of manipulating the robot while coping with the disaster mission and the training event by driving the robot.

In addition, the training evaluation module may derive a result by comparing it with pre-stored scenario success data using the driving information of the robot, the position information of the robot, and the time information when the robot passed the scenario in the scenario. .

The MR environment and robot interaction training system of the present invention for solving the above problems can create an item that can be trained through the virtual training event by allowing the trainee to recognize the virtual training event under the control of the robot. It has the advantage of maximizing training without preparing for an actual training event.

In addition, the present invention has an advantage that it can have a training effect like a real environment more realistically by showing a phenomenon in contact with a virtual training event and a visual effect related to the operation of the robot.

Effects of the present invention are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the description of the claims.

1 is a diagram illustrating a robot in an actual training space (A) and a robot in a scenario displayed to trainees in an MR environment and robot interaction training system according to an embodiment of the present invention;
2 is a diagram conceptually illustrating a fire suppression scenario and a rescuer rescue scenario in an MR environment and a robot interaction training system according to an embodiment of the present invention;
3 is a view showing the overall concept for building a mixed environment in the MR environment and the robot interaction training system according to an embodiment of the present invention;
4 is a diagram illustrating a signal flow diagram between a system, a trainee, and a robot to recognize a virtual training event in an MR environment and a robot interaction training system according to an embodiment of the present invention;
5 is a diagram illustrating an algorithm for a robot to recognize a virtual training event in an MR environment and a robot interaction training system according to an embodiment of the present invention;
6 is a conceptual diagram illustrating a robot driving for a virtual training event in an MR environment and a robot interaction training system according to an embodiment of the present invention;
7 is a conceptual diagram of a state in which a robot can recognize a virtual training event by a control command in an MR environment and a robot interaction training system according to an embodiment of the present invention;
8 is a view showing the training degree of trainees according to whether a disaster mission is virtual or real through a training evaluation module in an MR environment and robot interaction training system according to an embodiment of the present invention.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those of ordinary skill in the art to which the present invention pertains can easily practice the present invention.

In describing the embodiments, descriptions of technical contents that are well known in the technical field to which the present invention pertains and are not directly related to the present invention will be omitted. This is to more clearly convey the gist of the present invention without obscuring the gist of the present invention by omitting unnecessary description.

For the same reason, some components are exaggerated, omitted, or schematically illustrated in the accompanying drawings. In addition, the size of each component does not fully reflect the actual size. In each figure, the same or corresponding elements are assigned the same reference numerals.

The present invention will be described in detail with reference to the drawings.

1 shows a robot 100 in an actual training space A and a robot 100 in a scenario displayed to trainees in an interaction training system 300 of an MR environment and a robot 100 according to an embodiment of the present invention. 2 is a diagram conceptually showing a fire (G1) suppression scenario and a rescuer (G2) rescue scenario in the interaction training system 300 of the MR environment and the robot 100 according to an embodiment of the present invention, FIG. 3 is a view showing the overall concept for establishing a mixed environment in the interaction training system 300 of the MR environment and the robot 100 according to an embodiment of the present invention, and FIG. 4 is an MR environment according to an embodiment of the present invention. It is a signal flow diagram between the system 300, the trainee, and the robot 100 to recognize a virtual training event F in the interaction training system 300 of the environment and the robot 100, and FIG. 5 is a signal flow diagram according to an embodiment of the present invention The robot 100 is a diagram showing an algorithm for recognizing a virtual training event F in the interaction training system 300 of the MR environment and the robot 100, and FIG. 6 is an MR environment and the robot according to an embodiment of the present invention. It is a conceptual diagram showing the operation of the robot 100 for a virtual training event (F) in the interaction training system 300 of 100, and FIG. 7 is an interaction training between the MR environment and the robot 100 according to an embodiment of the present invention. It is a conceptual diagram of a state in which the robot 100 can recognize the virtual training event F by the control command 331 in the system 300, and FIG. 8 is an MR environment and the robot 100 according to an embodiment of the present invention. ) is a diagram showing the training degree of trainees according to whether the disaster mission (E) is virtual or real through the training evaluation module in the interaction training system 300 of FIG.

The present invention implements a real-world disaster situation as a mixed environment (MIXED-REALITY), so that the trainee who controls the real robot 100 trains in a virtual disaster situation with the MR environment and the robot 100 interaction training system 300 ), and the configuration of the system 300 may include a mixed environment implementation module 310 , a scenario module 320 , an interaction module 330 , a robot driving module 340 , and a control module 350 . .

The system 300 may be stored in the robot 100 , and may be stored in the controller 200 in which the trainee controls the robot 100 .

First, the robot 100 according to an embodiment of the present invention has a body, a moving part, and a camera 120, and a part of the body may include a communication arm 140 capable of communicating with a virtual environment.

In addition, since the robot 100 is equipped with various sensors therein, it is possible to generate information about the robot 100 .

The sensor provided in the robot 100 may include a position sensor, a communication sensor, a speed (including acceleration) sensor, and a photo sensor.

And it is premised that the robot 100 is moved in the training space A, which is an actual environment of the present invention, and in the real space, markers are formed on the ceiling and wall, and the robot 100 receives position information through the marker. can be transmitted, and the image captured by the robot 100 can also be transmitted in real time.

The sensor data and image data are transmitted to the system 300 in real time, and the trainee can drive the robot 100 through the controller 200 that can drive the robot 100 in the training space (A).

The controller 200 provides a control command 342 to the robot 100 so that a trainee can remotely control the robot 100 in a space separate from the training space A.

The controller 200 includes a joystick and a button, and may be displayed as a mixed environment so that virtual target objects G1 and G2 and a virtual training event F are generated at a specific location in the image data.

Also, the controller 200 may have a display configured to view the mixed environment.

Therefore, the trainee can train in the form of driving the actual robot 100 in the training space A while watching a mixed environment such as a disaster environment in the controller 200 .

Meanwhile, creating a mixed environment through the image data may be performed by the mixed environment realizing module 310 .

And the scenario module 320 is linked with the mixed environment implementation module 310, and by identifying the structure of the training space (A), the disaster mission (E) and the scenario are achieved between the starting location (P1) and the ending location (P3). A scenario including a target location in which the virtual target objects G1 and G2 corresponding to the target are disposed may be generated.

In addition, the fire G1 may be extinguished by communicating with the virtual target objects G1 and G2 through the communication arm 140 mounted on the robot 100 .

The interaction module 330 may generate size and location information of the virtual target objects G1 and G2 as well as the virtual training event F.

Specifically, the interaction determining unit 332 determines whether the robot 100 is adjacent to the virtual target objects G1 and G2 with information on the virtual target objects G1 and G2 disposed at the target position, and the interaction control unit Reference numeral 334 indicates that the virtual target objects G1 and G2 disappear from the scenario through the direction and time of the signal generated from the communication arm 140 in response to the virtual target object G1 and G2 information.

The above scenario may be implemented as a mixed environment.

Therefore, the trainee proceeds with the scenario while viewing the image implemented in the mixed environment in the controller 200. In this case, the robot 100 may be driven by receiving the trainee's control command 342 in the mixed environment.

On the other hand, the interaction module 330 checks the disaster mission (E) occurring in the mixed environment, and determines whether the robot 100 has reached the contact point in contact with the disaster mission (E) by the control command 342, and the contact A control command 331 for controlling the robot 100 is generated at a point.

Specifically, the interaction module 330 reacts as if there is a virtual training event F at the point of contact where the trainee drives the robot 100 through the control command 342 and the virtual training event F exists. In order to do so, a control command 331 for controlling the driving of the robot 100 may be generated before the manipulation command 342 at the contact position.

Through this, the trainee can confirm on the display as if the robot 100 was moving and collided with the wall, which is the virtual training event (F).

Here, it may be the robot driving module 340 that provides the control command 342 and the control command 331 to the robot 100 .

The robot driving module 340 may provide a control command 331 to the robot 100 at a point of contact where the robot 100 comes into contact with the virtual training event F, such as when the robot 100 collides with the real disaster mission E. .

At this time, the trainee's display is provided with a scenario of a mixed environment, so that the trainee can see it as if he had hit a real wall.

In this way, the control command 342 and the control command 331 are selected and connected to the robot driving module 340 provided to the robot 100, and the control provided so that the trainee can check the results of the operation of the robot 100 in the scenario A module 350 may be included.

The control module 350 provides a steering command 342 and a control command 331 from the robot driving module 340 to the robot 100 so that the robot 100 comes into contact with the disaster mission E, or comes into contact with the robot 100 and leaves. A state in which the trainee can provide the control command 342 to the robot 100 may be provided in the completed state.

That is, the robot 100 is to provide a steering timing so that the trainee can recognize that the collision with the virtual training event (F).

In addition, according to an embodiment of the present invention, the interaction

An embodiment of the present invention will be described in detail with reference to the following drawings.

1 shows a fire G1 scenario in which virtual target objects G1 and G2 are set as a fire G1.

In the scenario, the robot 100 must move from the starting position P1 to the target position where the virtual target objects G1 and G2 are formed to suppress the fire G1. A disaster mission E may be formed between the starting position P1 and the target position.

The disaster mission (E) corresponds to the wall and may be an actual training event (E) formed in the training space (A), or a virtual training event (F) that is not in the training space (A).

In order to achieve the scenario mission, the trainee avoids the disaster mission (E), moves to the target location, and performs training to extinguish the fire (G1).

In the fire (G1) scenario, as shown in FIG. 2 , as well as training to suppress the fire (G1), it can be provided as a scenario to rescue the person in need (G2).

As such, the scenario is not limited to the rescue of the fire (G1) and the rescuer (G2), and according to the disaster situation in the scenario module 320, a scenario including a fire (G1), a tornado, a rescue, a flood, etc. may be provided. , and the corresponding virtual target objects G1 and G2 may be provided in various ways through scenario construction.

In this embodiment, a fire (G1) scenario is selected and described.

In FIG. 2 , the scenario module 320 is divided into a starting location P1 and an ending location P3 where the scenario starts, and a fire G1 corresponding to the scenario mission between the starting location P1 and the ending location P3. Alternatively, the requestor G2 includes a target position formed by the virtual target objects G1 and G2.

The scenario module 320 may adjust the arrangement of the start position P1, the target position and the end position P3, and the disaster mission E with actual information about the training space A.

In this case, the scenario module 320 may provide a scenario in the mixed environment to the trainee together with the mixed environment implementation module 310 .

In order to provide such a structure to the trainee so that the trainee can have a training effect corresponding to the actual disaster environment in a mixed environment, it can be implemented by the configuration of the system 300 of the present invention.

In addition, in the trainee evaluation according to the present invention, the scenario module 320 may randomly arrange an actual training event (R) and a virtual training event (F) on the scenario.

This, in conjunction with the training evaluation module 360, determines the number of disaster missions (E) according to the level of training, and may alternately place the disaster missions (E) in virtual or virtual, or select only one of the virtual or virtual can also be placed.

And the disaster mission (E) may include an event (not shown).

The event can be implemented in virtual or real form, and may include an area where oil leaks on the floor where the robot is driven, gusts of wind, rain, hail, thunder, and gravel spread in the scenario.

Since an object of the present invention is to allow a trainee to experience virtual target objects G1 and G2 implemented virtually in a mixed environment as an entity, details will be described with reference to FIGS. 3 and 4 .

In FIG. 3 , the trainee driving the robot 100 with the controller 200 intervenes in the system 300 to control the robot 100 drive and implement the mixed environment, and the robot 100 is moved in the training space A, but Trainees can experience it just like the real thing.

As shown, the controller 200 has two displays.

The display may include a first display (B) that can confirm that the robot 100 is driven in a scenario in a mixed environment, and a second display that shows the current stage of the robot 100 in all stages of the scenario.

The state data indicating the speed and coordinate values of the robot 100 and damage information may be displayed on the first display (B).

First, the mixed environment implementation module 310 implements a mixed environment by creating a virtual training event (F) and virtual target objects (G1, G2) in the image through the image captured by the robot 100 and the information of the training space (A). and the virtual training event F and the virtual target objects G1 and G2 are displayed in real time in response to the image in which the robot 100 is moved and the direction is changed.

Here, the mixed environment implementation module 310 may arrange the virtual training event F and the virtual target objects G1 and G2 together with the scenario module 320 .

In addition, the interaction module 330 determines the location of the disaster mission E in the scenario and generates a control command 331 capable of controlling the robot 100 at the contact point.

Specifically, the interaction module 330 is an actual training event (E) in a training space (A) that is a disaster mission (E) or a virtual training event (F) generated by the mixed environment implementation module 310 or a disaster mission (E) It may include an interaction determination unit 332 that determines the type of , and generates disaster mission (E) information including the location of the disaster mission (E) in the scenario.

In addition, the disaster mission (E) information for the virtual training event (F) adjacent to the robot 100 is provided to the robot 100 to control the robot 100 when the robot 100 is close to the disaster mission (E). It may include an interaction control unit 334 that generates a control command 331 and provides it to the robot driving module 340 .

The interaction determination unit 332 generates the size of the virtual training event (F) as a plurality of coordinate data on the scenario according to the disaster mission (E) information to measure the distance between the robot 100 and the virtual training event (F). The interaction control unit 334 provides the control command 331 to the robot driving module 340 at the contact point where the robot 100 and the virtual disaster mission E come into contact, so that the robot driving module 340 is the robot ( 100) may be able to provide a control command 331 .

The control command 331 may include coordinate data for a virtual volume of the disaster mission E according to the disaster mission E information.

Subsequently, in order for the interaction control unit 334 to provide the control command 331 to the robot driving module 340, it is determined as a preset threshold value between the robot 100 and the virtual training event F, and the robot 100 is contacted. The control command 331 may be provided at a threshold imminent at a point.

At this time, the trainee continues to transmit the control command 342 , but the robot driving module 340 preferentially provides the control command 331 to the robot 100 at the contact point.

In addition, the interaction module 330, the robot 100 receives the control command 331, the robot 100 is in contact with the virtual training event (F) to the control module 350 the effect that the trainee can visually confirm that It may include an interaction effect unit 336 to provide.

The interaction effect unit 336 is a visual effect in which the robot 100 collides with the virtual training event F when the robot 100 sequentially receives the control command 342 while receiving the control command 331 . It is provided to the control module 350, and at this time, the control command 342 is analyzed to determine the speed of the robot 100 and the number of times the robot 100 has contacted or collided with the virtual training event (F). It is connected to the mixed environment implementation module 310 so as to be transformed.

Let me explain with an example.

Assume that the robot 100 is in contact with the virtual training event F several times due to the trainee's manipulation mistake.

The interaction determination unit 332 first generates the disaster mission (E) information of the virtual training event (F), and the interaction control unit 334 is the robot 100 and the location of the robot 100 in proximity to the disaster mission (E). When it is determined that is a threshold value, the control command 331 is provided to the robot driving module 340 . In addition, the robot driving module 340 provides a control command 342 to the robot 100 so that the robot 100 is driven, and when the control command 331 is received, control when the robot 100 is located at the contact point. A command 331 is provided to the robot 100 , and when it deviates from the contact point, a control command 342 is provided to the robot 100 again.

The control command 331 causes the robot 100 to recognize the virtual training event F. Therefore, when the control command 331 is received, the control command 342 is not received, so that the trainee can feel it as if the robot 100 hit a wall.

The reason why the interaction effect unit 336 visually implements that the robot 100 collides with the virtual wall is because the interaction effect unit 336 is connected to the mixed environment implementation module 310 .

The interaction effect unit 336 visually expresses that the robot 100 collides with the virtual training event F at the contact point through the control command 331. The effect image is the speed and number of collisions of the robot 100. Accordingly, an effect image in which the degree of damage to the disaster mission E is distinguished may be provided to the control module 350 .

The sound and vibration according to the collision may be generated so that the trainee is aware of the real environment.

In addition, the interaction effect unit 336 receives driving information through various sensors mounted on the robot 100 , and determines the control command 342 for driving the robot 100 to determine the driving state of the robot 100 . can be expressed visually.

The interaction effect unit 336 receives the manipulation command 342 through the robot driving module 340, extracts driving information from the sensor of the robot 100, and reacts the robot's rapid acceleration, rapid rotation, and contact point. It can be expressed as an entity.

For example, if the robot 100 is in a position with oil in the scenario, the robot 100 receives the forward steering command 342 and proceeds forward, but the robot 100 moves forward through a virtual object called oil. It can be expressed to increase the relative rotation speed of the wheel.

This means that the interaction effect unit 336 understands the relationship between the virtual training event and the robot through the mixed environment implementation module 310 and predicts the robot's response in the virtual training event (F) and the environmental disaster mission (not shown). It can be a method of realizing a pre-stored mixed environment when the robot is in the corresponding position by creating

Although oil has been described as an embodiment in this embodiment, it may be in a form in which the environmental form of wind and the floor is modified, and the mixed environment implementation module 310 transforms the shape of the robot and the virtual training event according to the virtual training event. In terms of providing it to the control module 350, it may be reflected in various embodiments.

Meanwhile, the state when the robot 100 comes into contact with the disaster mission E is shown in FIG. 4 as a flow according to the signal.

When a trainee first controls the robot 100 in a situation where the robot 100 departs from the starting position P1, the robot driving module 340 provides a control command 342 to the robot 100, and the robot 100 The image of is transmitted to the system 300 .

In the system 300 , the mixed environment implementation module 310 corresponds to the image and provides the mixed environment to the display, and the trainee continues to control the robot 100 and moves to a target position.

When the robot 100 encounters a virtual training event (F) while moving, the interaction determination unit 332 generates disaster mission (E) information and provides it to the interaction control unit 334, and the interaction control unit 334 provides a control command 331 ) can be generated and provided to the robot driving module 340 and inform the interaction effect unit 336 that the robot 100 contacts or collides with the contact point. And the robot driving module 340 allows the robot 100 to provide a mixed environment with the result corresponding to the virtual training event F through the control module 350 . After the virtual training event F, the trainee continues to move by controlling the robot 100 .

The robot driving module 340 simultaneously receives the trainee's control command 342 and the control command 331 of the interaction control unit 334, and when the robot 100 is positioned above the threshold of the contact point, the control command 331 to the robot 100 to make the robot 100 mistake it for recognizing the virtual training event F.

At this time, the control command 331 is given priority over the manipulation command 342 , and the control command 331 is to generate a signal of the virtual training event (F).

If the control command 342 is continuously generated to drive the robot 100, the control command 331 moves the robot 100 to a position corresponding to the size of the disaster mission E through the virtual training event F information. It is driven by the control command 331 so that trainees who control the robot 100 have a feeling as if they hit an actual wall.

And the interaction effect unit 336 is connected to the interaction control unit 334, when the control command 331 is generated, it determines the speed and the number of collisions of the robot 100 to control the image in which the virtual training event (F) is damaged Provided together with module 350 , the trainee can visually confirm that the disaster mission E is damaged in response to a collision.

In a situation where the virtual training event F and the robot 100 are close to each other, the interaction module 330 operates as shown in FIG. 5 .

When the steering command 342 is received, the robot 100 is driven (S01) through the steering command 342, and the steering command 342 generates the disaster mission (E) information by the interaction determination unit 332 and controls the command. It is determined whether 342 has reached the contact point ( S02 ), and it is determined whether the robot 100 has reached the contact point and made contact with the virtual training event F ( S03 ).

And if the robot 100 reaches the contact point and does not come into contact with the virtual training event F, the robot driving module 340 provides a control command 342 to the robot 100, and if it has contacted, the robot driving module 340 ) provides the control command 331 to the robot 100 (S04), and the robot continues to receive the control command 342 while receiving the control command 331 to receive a virtual training event (F) or a virtual training event (F). The effect of hitting the wall F is provided to the trainee (S05), and the control module 350 is displayed on the display (S06).

A state in which the control command 331 enables the robot 100 to come into contact with the virtual training event F is shown in FIGS. 6 to 7 .

Since there is no virtual training event F in the training space A, the user may pass through the virtual training event F due to a steering mistake.

However, the control command 331 controls the driving force of the robot 100 so that the virtual training event F and the robot 100 can come into contact with each other to prevent the virtual training event F from passing.

Since the interaction determination unit 332 generates the disaster mission (E) information as coordinate data, the control command 331 controls the operation of the robot 100 in response to the size and shape of the virtual training event F.

It is an object of the present invention to increase the training element of the trainee by recognizing the virtual training event F by the robot 100 as well.

Evaluating whether the trainee has performed the scenario mission to avoid the disaster mission (E) in the scenario is performed in the training evaluation module 360 .

First, the elements of training in a scenario can be a disaster mission (E) and a scenario mission.

Accordingly, the trainee is evaluated by the training evaluation module 360 in a series of processes of overcoming the disaster mission E and completing the scenario mission.

The evaluation may be a score evaluation in units of 1 point on a scale of 1 to 10, or success or failure.

For example, if the robot 100 is driven by the control command 342 and then the control command 331 is generated, a loss may occur.

In addition, disaster missions (E) and events can be evaluated relative to other trainees as time out of the situation.

To this end, the training evaluation module 360 may include a separate display through which the evaluator can check the screen for driving the trainee's robot 100 together.

As shown in FIG. 8 , if an actual disaster mission R is placed in the training space A in the training evaluation module 360 , the robot 100 is placed on the actual disaster mission R according to the steering command 342 . Although contact is possible, it can be an item of training, but implementing the disaster mission (E) similar to the actual situation is difficult to implement because it incurs a lot of cost and the number of training items is inevitably reduced because the number is limited.

However, with the intervention of the system 300, when the robot 100 recognizes the virtual training event (F), the trainee can implement the virtual training event (F) in various ways, and various training items can be arranged on the scenario Do. Therefore, the number of training items increases, and the evaluation items can be evaluated according to the training items.

Training of trainees is carried out in a scenario, and it is possible to evaluate the achievement of the scenario mission and the ability to cope with the disaster mission.

Specifically, in the scenario, trainees cannot distinguish between an actual training event (R) or a virtual training event (F), and both are perceived as a disaster mission (E).

Therefore, when the disaster mission (E) appears, the robot 100 can be driven with the control command 342 to avoid the disaster mission (E), and the disaster mission (E) and the disaster mission (E) cannot be avoided. If it is touched, it may be a factor of deduction of points.

Also, when the scenario task is a fire, the fire can be extinguished by operating the communication arm 140 and communicating with the virtual fire.

At this time, the distance between the robot 100 and the fire, the time to operate the communication arm 140 when operating, etc. are elements of training, and the trainee is trained through the trainee's control command 342 in the training evaluation module 360. can be evaluated

As described above, preferred embodiments according to the present invention have been reviewed, and the fact that the present invention can be embodied in other specific forms without departing from the spirit or scope of the present invention in addition to the above-described embodiments is one of ordinary skill in the art. It is obvious to them. Therefore, the above-described embodiments are to be regarded as illustrative rather than restrictive, and accordingly, the present invention is not limited to the above description, but may be modified within the scope of the appended claims and their equivalents.

100: robot
200: controller
300: system
310: Mixed Environment Implementation Module
320: scenario module
330: interaction module
332: interaction judgment unit
334: interaction control
336: interaction effect unit
340: robot driving module
350: control module

Claims (9)

As a training system for training that trainees can respond to disaster missions by controlling robots in a disaster situation,
The trainee controls the robot to set an environmental factor that may occur during the disaster mission performance as a training event, and the trainee drives the robot to perform the disaster mission and to train in response to the training event Scenario module to build a scenario so that;
a mixed environment realization module for realizing the disaster mission in a virtual or real form in the scenario;
In the scenario, it is determined whether the training event is real or virtual, and when the virtual training event occurs while the trainee is manipulating the robot to perform the disaster mission, the trainee precedes the command to control the robot. an interaction module for causing the robot to restrict the operation of the robot due to the virtual training event so that the trainee recognizes the virtual training event;
a robot driving module that, when the operation of the robot is restricted in the interaction module, prioritizes a command for restricting driving of the robot over a command for the trainee to control the robot and sends the command to the robot; and
And a control module that provides the trainee with a result of driving the robot by a command of the trainee while performing the disaster mission and a result of limiting the operation of the robot in the interaction module,
The interaction module provides, to the control module, a change in the robot and a change in the virtual training event in response to a result of limiting the driving of the robot in the virtual training event,
The interaction module,
In the scenario, it is determined whether the training event is an actual training event or a virtual training event. an interaction determination unit that determines whether they match;
an interaction control unit receiving the determination result from the interaction determination unit and providing a command to limit the operation of the robot to the robot driving module when the trainee fails to control the robot to respond to the virtual training event; and
A signal stimulating the senses of the trainee is provided to the control module of the state of the robot whose operation is restricted by the virtual training event through the interaction control unit to respond to the result of the trainee limiting the operation of the robot. Characterized in that it comprises an interaction effect unit for recognizing a change and a change of the virtual training event,
MR environment and robot interaction training system.
delete According to claim 1,
The interaction module,
When the interaction determining unit determines that the training event is the actual training event, the interaction control unit is not involved in driving the robot,
MR environment and robot interaction training system.
According to claim 1,
The interaction control unit,
Extracting the training event information from the scenario module to generate a control command that is a command for limiting the operation of the robot in response to the size and location of the virtual training event
MR environment and robot interaction training system.
According to claim 1,
The interaction effect unit,
The trainee analyzes the command to control the robot, and mixes the deformed state of the robot and the deformed state of the virtual training event according to the speed of the robot and the number of times the robot has contacted and collided with the virtual training event characterized in that it is expressed in the environment,
MR environment and robot interaction training system.
5. The method of claim 4,
The robot is
Characterized in that it comprises a robot action unit that physically changes the robot state of the robot so that the trainee can recognize a result corresponding to the virtual training event,
MR environment and robot interaction training system.
7. The method of claim 6,
The robot is provided with wheels to be movable,
The robot action unit,
It is mounted on the wheel of the robot, characterized in that it blocks the driving force to rotate the wheel by the control command,
MR environment and robot interaction training system.
According to claim 1,
The trainee further comprises a training evaluation module for evaluating the result of manipulating the robot while coping with the disaster mission and the training event by driving the robot,
MR environment and robot interaction training system.
9. The method of claim 8,
The training evaluation module,
In the scenario, the robot's driving information, the robot's position information, and the time information when the robot passed the scenario are compared with pre-stored scenario success data to derive a result,
MR environment and robot interaction training system.

Images