KR101850410B1 - Simulation apparatus and method for teaching robot based on virtual reality - Google Patents

Abstract

The present invention relates to a simulation device and a method for robot teaching based on virtual reality. More specifically, the simulation device and the method for the robot teaching based on the virtual reality realize a real working environment of a robot implemented in the virtual reality through three-dimensional modeling. In addition, robot models in the virtual reality are supported to interact with each other according to operation information of a user provided through virtual reality equipment of the user for the robot operation teaching. Moreover, an optimal motion path in which operation according to the operation teaching of the robot models implemented in the virtual reality is performed without collision is calculated, and the optimal motion path is provided to the real robot.

Description

Technical Field [0001] The present invention relates to a virtual reality-based robot teaching apparatus,

The present invention relates to a simulation apparatus and method for teaching a virtual reality-based robot, and more particularly, to a virtual reality system that realizes a real working environment of a robot in a virtual reality through three-dimensional modeling, The virtual robot 100 includes a plurality of virtual robots 120 and a plurality of virtual robots 120. The virtual robot 120 includes a plurality of virtual robots, And to a simulation apparatus and method for teaching a virtual reality-based robot.

In addition to the development of robotic technology, there is an automatic process using robots in various work environments, thereby improving work efficiency and productivity of the process.

A programming method for generating a series of motion sequences for a robot to perform a task in a specific work environment can be divided into two types. One is an OLP (off-line programming) method using software, the other is a teaching method This is the case.

The OLP method generates a work file for a robot's work path based on three-dimensional modeling information about the work site, CAD information of the work, and information of the robot. It is impossible to support the operation of the robot intended by the user to interact with the operation of the user so that a considerable time is required to set the operation of the robot and an inefficient operation path is set in the robot frequently There is a problem.

In addition, since the OLP method sets the operation range of the robot that can operate in the work environment and the work environment in advance, the work environment is re-modeled through the existing general rendering method when the work environment is changed, Therefore, there is a problem that it is difficult to utilize in a place that is not a highly standardized work environment because of the time and effort required for such setting.

Therefore, the existing OLP method is required to diversify products and develop new products continuously in order to secure corporate competitiveness, and it is difficult to cope with the recent situation where the work environment continuously changes.

Recently, teaching method by Teach Pendant has been used to solve the problem according to the existing OLP method, and this teaching method is very useful for creating work for a robot having a certain degree of freedom in the field, It is not easy for the operator to generate motion in a condition where the degree of freedom is high like the robot and the working space of both arms is overlapped and the collision is likely to occur.

In addition, since most of the robot joints are rotating, it is very difficult to map the control commands and robot motion of the teach pendant when creating an action such as collision avoidance of the robot. have.

In addition, since the processing time is significant in the process of mapping the teach pendant to the operation of the robot, it takes a considerable time for the user to set the operation of the robot, so that the real-time performance and responsiveness of the robot in response to the user's operation is significantly lowered The efficiency of the robot teaching is deteriorated excessively.

In addition, the teach-in method using the teach pendant is also complicated and requires a considerable amount of time to set up the robot operation because a specialist visits each time the work environment changes in a dynamic work environment in which a change may occur, As a result of the mapping error between the control command of the teach pendant and the robot operation, it is difficult to easily detect the collision between the robot and the obstacle occurring in the work environment or the robot's own collision, So that it is difficult to set up an efficient operation of the robot.

In recent years, in addition to the above-mentioned methods, a direct teaching method for generating an operation intuitively by moving an operator directly has appeared, and a non-expert can easily and quickly create a robot operation by overcoming the problems of the conventional method using the teach pendant There is an advantage.

However, in order to perform the direct teaching in this way, the function of the robot to adaptively move by reflecting on the external force of the human being is necessarily required. However, since the conventional industrial robot does not satisfy most of them, there is an inconvenience to install a separate sensor on the outside.

In addition, for direct teaching, the worker must create a robot task at the actual work site where the robot is located. However, if the robot work site is not accessible to the human, this method can not be applied. It can be very dangerous to the operator if it raises.

Korean Patent No. 10-1052740

In order to solve the above-described problems, the present invention provides a method for realizing a real working environment of a robot in a virtual reality through three-dimensional modeling, Accordingly, it is possible to calculate the optimal operation path of the virtual robot model implemented in the virtual reality, which is performed without collision, and to set it in the robot so as to interact with the user's operation, It supports the convenience of the robot's operation setting by supporting the interaction of the robot, and intuitively confirms the robot's operation through the virtual reality. Also, it supports the robot's optimized operation path To be automatically generated and set up.

In addition, the present invention models the actual work environment at a level that can detect collision, thereby dramatically reducing the computation time and load required for modeling the work environment, and at the same time, The posture in which the robot itself collides with the robot can be easily detected in accordance with the collision between the robot and the obstacle configured in the working environment or the posture of the robot on the basis of the model and reflected on the coordinate space so that the robot can respond to the user's teaching The purpose of this study is to support the robot to create an optimal motion path of the robots without collision.

In addition, the present invention generates control information for the robot's operation corresponding to the user's instruction in the virtual reality, and supports the robot to set it conveniently so that the robots located in dangerous work places can be reliably guided to the user's teaching And it is an object of the present invention to support the operation setting of the corresponding robot.

A simulation apparatus for teaching a virtual reality-based robot in cooperation with a user's virtual reality interface apparatus for teaching a robot according to an embodiment of the present invention includes a point-of- A work environment modeling unit for modeling plane clusters based on coordinates of depth images obtained on the basis of the depth information according to the point group information to generate plane model clusters based on polygons or hexahedrons to generate environment model information for a reduced work environment; A robot operation modeling unit that forms a tree structure for calculating the operation path of the robot by filling a predetermined coordinate space in which individual attitudes of the robot are set as coordinates with arbitrary samples by a Poisson disk sampling method, Based on the preset robot model information for the robot, A virtual reality interface device for generating an image of a real interface and receiving teaching information including a current position and a target posture of a user for robot teaching from the virtual reality interface device, Wherein the control unit forms the tree structure in which the posture of the robot in which the collision occurs is excluded based on the environment model information and the robot model information in cooperation with the work environment modeling unit, And a control unit for calculating an optimal path for moving the robot to the robot posture and providing the generated image by reflecting the optimal path to the robot model information.

As an example related to the present invention, the work environment modeling unit may check the planarity of each cluster, and may classify the non-planar cluster until the plane is confirmed.

According to an embodiment of the present invention, the work environment modeling unit may include an input unit for receiving the point cloud information generated by scanning the work environment of the robot, and an input unit for clustering based on the coordinates of the depth image obtained based on the depth information according to the point cloud information And a data matrix of a point group belonging to an image coordinate region of each cluster is constructed and it is determined whether or not the plane is a plane through the rank of the corresponding data matrix to judge whether the cluster is flat or not, And a modeling unit for modeling the planar determination unit and the clusters determined by the plane as planar polygons or hexahedrons to reduce the amount of data.

In one embodiment of the present invention, the control unit maps the individual posture of the robot at the time of generating the optimal path, and controls the robot to interact with the operation of the user according to the teaching information based on preset robot model information modeled by the robot And an operation of the robot model according to the robot model information according to the optimal path is generated as an image and transmitted to the virtual reality interface device.

As an example related to the present invention, the control unit detects a collision between the robot and the working environment or whether the robot itself collides with the individual pose of the robot based on the environment model information and the robot model information, Wherein the robot movement modeling unit sets the limited space or the limited space corresponding to the collision posture in the coordination space by collecting the collision postures to the arbitrary sample And the like.

In one embodiment of the present invention, the controller calculates the optimal path based on a predetermined probabilistic roadmap method (PRM) in the optimal path calculation.

In one embodiment of the present invention, the robot operation modeling unit selects the arbitrary sample arbitrarily selected in the coordinate space by the Poisson disk sampling method, and repeats the sampling process to sample the entire coordinate space And a tree generating unit for forming a tree structure based on the selected sample.

According to an embodiment of the present invention, the control unit may generate control information for controlling the operation of the robot on the basis of the optimum path corresponding to the user's teaching.

The control unit may transmit the control information to the robot through the communication unit to perform a task operation corresponding to the optimum path corresponding to the control information, Is set in the robot.

The virtual reality-based robot teaching method of the simulation apparatus interlocked with the user's virtual reality interface apparatus for teaching the robot according to the embodiment of the present invention is characterized in that the point cloud information generated by scanning the work environment through the three- Generating environment model information for a reduced working environment by modeling plane clusters in a polygonal or hexahedron by clustering based on coordinates of depth images obtained based on the depth information according to the point group information, In the preset coordinate space in which the individual postures of the robots are set to coordinates, the rest of the robot except for the posture of the robot where the collision occurs is filled with a random sample by a Poisson disk sampling method on the basis of the environment model information and the robot model information A tree structure for calculating the operation path of the robot Generating an image for a virtual reality interface based on the environment model information and preset robot model information for the robot, providing the virtual reality interface through the virtual reality interface device, Generating an optimal path for moving the robot from the current posture to the target posture based on the teaching information and the tree structure, To the robot model information, and providing the generated image.

The present invention provides a virtual reality image by implementing a working environment of a robot and a robot as a virtual reality and provides a virtual reality image through a robot model interacting with a user's operation while the user confirms the operation of the robot in real time on the virtual reality In addition to supporting the intuitive direct teaching by supporting the teaching, it is possible to automatically exclude the posture where the collision with the obstacle located in the working environment occurs and the self collision of the robot in the posture of the robot, It is possible to calculate the optimal operation path of the robot which does not cause the collision in correspondence with the teaching of the operation motion of the robot, so that the operation operation of the robot corresponding to the stable and accurate teaching of the user can be set in the robot, It is effective to support the instruction to be performed easily.

In addition, the present invention compresses the point cloud information generated by scanning the working environment to a level that can detect whether or not the robot is in conflict with the robot during the working environment of the robot by using a hexahedron or polygon, The amount of data of the work environment model can be greatly reduced, and the load and computation time required for judging the collision of the robot in the work environment can be greatly reduced. Thus, the optimal operation path of the robot corresponding to the user's teaching is provided in real time At the same time, it is possible to guarantee immediate response of the robot model in response to the user's operation.

In addition, according to the present invention, in the sampling process of selecting a plurality of different postures that can be taken by a robot in a coordinate space in which a posture of a robot is formed as a coordinate system based on a probabilistic roadmap method, The selected sample in the coordination space can be uniformly distributed. Therefore, the optimum motion path among the various robot motion paths is automatically calculated to move from the current posture of the robot to the target posture set by the user according to the teaching of the user Thereby increasing the efficiency of the operation of the robot and optimizing the distribution of the sample to fill the coordinate space with the smallest number of samples to increase the operation speed for the calculation of the optimum operation path, Shorten the time required to set the operation, Real - time performance and responsiveness of the robot model according to the instruction.

In addition, the present invention provides an advantage of intuitive direct teaching and supports direct teaching in a virtual reality so that the operator does not have to go directly to the scene, thereby overcoming the physical limitations of the space, It is possible to directly create a work, and there is no need for a worker to directly work in a dangerous environment, thereby ensuring worker safety.

1 is a block diagram of a simulation apparatus for teaching a virtual reality-based robot according to an embodiment of the present invention;
FIG. 2 and FIG. 3 are operation configuration diagrams of a simulation apparatus for teaching virtual reality-based robots according to an embodiment of the present invention;
FIG. 4 is an exemplary view illustrating a clustering process of a work environment modeling unit according to an embodiment of the present invention; FIG.
FIG. 5 and FIG. 6 are diagrams illustrating an operation example for determining whether or not the work environment modeling unit according to an embodiment of the present invention is planar.
FIG. 7 and FIG. 8 illustrate an operation example of a modeling process of a work environment modeling unit according to an embodiment of the present invention; FIG.
FIG. 9 is an exemplary view of a work environment model generated through modeling of a work environment modeling unit according to an embodiment of the present invention; FIG.
FIG. 10 and FIG. 11 are exemplary views of a sampling process of the robot operation modeling unit according to the embodiment of the present invention. FIG.
FIG. 12 is an exemplary diagram illustrating calculation of an optimal operation path of a robot operation modeling unit and a control unit according to an embodiment of the present invention; FIG.
13 is a flowchart illustrating a simulation method for teaching a virtual reality-based robot according to an embodiment of the present invention.

Hereinafter, detailed embodiments of the present invention will be described with reference to the drawings.

1 is a block diagram of a simulation apparatus 100 for teaching virtual reality-based robots according to an embodiment of the present invention. As shown in FIG. 1, a work environment modeling unit 110, a robot operation modeling unit 120, (130).

In addition, a simulation apparatus (hereinafter, a simulation apparatus) 100 for teaching virtual reality-based robots can interoperate with a virtual reality interface apparatus 20 (or a virtual reality apparatus) that recognizes a user's operation and generates operation information The virtual reality interface device 20 includes an HMD (Head Mounted Display) device worn by a user, a controller or a teach pendant for recognizing a user's operation for teaching a robot And the virtual reality interface device 20 may include a plurality of different devices such as a device for providing (displaying) images and a device for recognizing the operation of a user.

The simulation apparatus 100 may further include a separate interface unit or a communication unit connected to the virtual reality interface unit 20. [

In addition, the virtual reality interface device 20 may generate operation information on the user's operation according to the user's operation or automatically recognize the operation of the user, generate the operation information, and transmit the generated operation information to the simulation device 100.

Based on the above-described configuration, the detailed operation of the simulation apparatus 100 will be described with reference to FIGS. 2 and 3. FIG.

2, the work environment modeling unit 110 scans the work environment of the robot to generate three-dimensional scan information for the work environment from the three-dimensional sensor 30, Scan information can be received.

At this time, the 3D scan information may be composed of three-dimensional point-cloud information (or point cloud data) for the work environment, and the work environment modeling unit 110 may include depth information And a predetermined image coordinate system may be applied to the depth image.

In addition, the work environment modeling unit 110 may cluster the point cloud information into block shapes on the basis of image coordinates according to an image coordinate system applied to the depth image, thereby creating a plurality of different clusters.

At this time, the plurality of different clusters may have the same size.

The work environment modeling unit 110 may determine whether the cluster is plane based on the point cloud information belonging to the cluster through a predetermined algorithm for each cluster, The information can be modeled by compressing it into a single plane polygon or a flat hexahedron.

In addition, the working environment modeling unit 110 may divide the non-planar cluster into a plurality of different detailed clusters, determine whether the divided clusters are planar according to the algorithm, and determine whether the planar clusters are polygonal The clusters may be divided until the planes are confirmed by repeating the process of modeling by compressing them into hexahedrons, dividing the divided clusters not the planes again to determine whether they are planar, and compressing them into the polygons or the hexahedron.

Accordingly, the work environment modeling unit 110 compresses the point cloud information belonging to each cluster corresponding to the work environment into a polygon or a hexahedron, and models the work, thereby generating a work environment based on the point cloud information on the work environment, It is possible to reduce the amount of data of the environment model information and greatly reduce the calculation time required for modeling the working environment, thereby supporting rapid modeling of the working environment.

In addition, the work environment modeling unit 110 may provide the work environment information to the controller 130.

Meanwhile, the controller 130 may store robot model information modeled as a robot to be taught in advance, and may include a robot that operates in a real working environment based on the robot model information and environment model information, A virtual reality interface related virtual reality interface image can be transmitted to the user's virtual reality interface apparatus 20 through the interface unit or the communication unit.

At this time, the virtual reality image may be composed of three-dimensional graphic information, and various interface-related information necessary for the robot operation may be included in the corresponding virtual reality image.

In addition, the robot model information may include information that the individual posture of the robot is mapped to the robot model.

Thus, the user can intuitively grasp the real working environment of the robot and the robot through the virtual reality.

Meanwhile, the control unit 130 may receive operation information for operating the robot from the operation recognition apparatus of the user by a user's intended operation, recognize the posture of the robot on the basis of the operation information, (Virtual robot) according to the model information so as to interact with the operation posture of the user, and based on the robot model information set in the posture of the robot corresponding to the operation information and the virtual reality image And provide the real-time information to the user's virtual reality interface device 20.

Accordingly, the virtual reality interface device 20 displays images through the virtual reality images so as to change the posture of the robot model corresponding to the user's operation posture, thereby enabling the user to intuitively grasp the motion of the robot, It is possible to ensure the convenience of setting the work contents of the robot by making the actions of the robots interact with each other.

In addition, the controller 130 can receive gaze information about the user's gaze direction from the virtual reality interface device 20, and based on the environment model information and the robot model information, And the robot model corresponding to the robot model information may be generated as the virtual reality image and may be provided to the virtual reality interface device 20. [ At this time, the gaze information may be included in the operation information.

The controller 130 generates control information on the operation of the robot intended by the user based on the operation of the robot interacting with the user's operation in the virtual reality, However, it is possible to determine whether the robot itself collides with a robot, such as a collision between the robot and an obstacle located in the working environment, or a collision between the arms of the robot during the operation of the robot through the virtual reality interface So that the control information can be generated in a state in which the attitude of the robot in which the collision occurs is included in the user's teaching process.

Therefore, the present invention can avoid such a collision to automatically generate the most efficient robot operation operation corresponding to a work operation intended by the user, which will be described in detail based on the configuration of the controller 130 of FIG. 1 .

The control unit 130 may include an operation recognition unit 131, an interaction detection unit 132, and an image generation unit 133, as shown in FIG.

First, the virtual reality interface device 20 automatically recognizes a user's operation according to a user's input in order to instruct a robot about a work operation, so that the current position, which is a posture in which a work operation of the robot starts, It is possible to generate the teaching information on the target attitude, which is the ending attitude, and transmit the teaching information to the simulation apparatus 100.

For example, the virtual reality interface device 20 can recognize the position and direction of the user's head and both hands, and the position and orientation information of the head can be used to control the viewpoint of visualization in the virtual reality, May be used for interaction with the robot, and information on the information may be included in the above-described teaching information and operation information. In addition, the virtual reality interface device 20 may be equipped with a button for recognizing a user event to generate the teaching information.

The virtual reality interface device 20 moves the end point of the robot model on the virtual reality to move the robot so that the user moves his / her hand to the end-effector of the robot, The robot can move the point of action of the robot by moving the hand from the current position to a desired position and direction.

At this time, the virtual reality interface device 20 determines the posture of the user when the action point is constrained (when an event occurs) as the current posture (or start posture) of the robot where the robot starts teaching, can do.

In addition, the virtual reality interface device 20 calculates the distance between the position of the user's hand and the point of action of the robot model, detects events such as whether the user confirms the button input when the distance is within the effective distance and the user constrains the robot end point, The teaching can be terminated.

At this time, the worker can move the point of action of the robot model on the virtual reality, but also can directly control the motion of each joint constituting the robot model.

Accordingly, when an event corresponding to the teaching end is detected, the virtual reality interface device 20 can generate posture information on the target posture of the robot model corresponding to the posture of the user corresponding to the event .

Then, the virtual reality interface device 20 can transmit the teaching information including attitude information about the current attitude and the target attitude to the controller 130 of the simulation apparatus 100.

At this time, the virtual reality interface device 20 may generate attitude information for each of the current attitude and the target attitude as individual teaching information, and may transmit the attitude information to the controller 130 of the simulation apparatus 100. In other words, the virtual reality interface device generates posture information about the current posture as teaching information, transmits the posture information to the control unit 130, generates posture information about the target posture as teaching information, and transmits the generated posture information to the control unit 130 .

Accordingly, the motion recognition unit 131 included in the control unit 130 can receive the teaching information, and can recognize the current posture and the target posture of the robot model according to the teaching information. The motion recognition unit 131 may provide information on the current posture and the target posture according to the teaching information to the interaction detection unit 132 included in the control unit 130. [

At this time, the motion recognizing unit 131 may receive input information such as an operation information about an operation of a user and an event for restricting the action point from the virtual reality interface apparatus 20, It is possible to generate teaching information on the current posture and the target posture instead of the virtual reality interface device 20 on the basis of the present invention.

That is, since the virtual reality image is generated based on the preset robot model information, the motion recognition unit 131 recognizes that the operation information and the input information generated based on the virtual reality interface- It is possible to easily recognize (detect) the attitude of the robot model mapped to one attitude from the robot model information, and generate the teaching information based on the attitude.

The virtual reality interface device 20 may cancel at least one of the current attitude and the target attitude and may reset the information. When the information is transmitted to the controller 130 of the simulation apparatus 100, the controller 130 May delete the teaching information corresponding to the posture in which the cancellation has occurred and reset at least one of the current posture and the target posture according to the newly received teaching information.

Meanwhile, the interaction detecting unit 132 included in the control unit 130 determines whether or not an obstacle, which is configured in the environmental model according to the environmental model information, is detected with respect to the individual attitude that the robot model can take based on the environment model information and the robot model information Such as an attitude in which a collision with the robot is caused or a collision between the arms of the robot, such as a posture in which the robot may collide with itself, can be detected, and the robot motion modeling unit 120 as collision posture information.

When the posture corresponding to the motion information received from the virtual reality interface device 20 corresponds to the collision posture in cooperation with the motion recognition unit 131, To the virtual reality interface device 20, and the virtual reality interface device 20 can display the corresponding notification information through the virtual reality image.

On the other hand, the robot operation modeling unit 120 may include a coordinate system (coordinate system) in which the individual postures of the robots can be displayed in coordinates (the individual postures of the robots are configured as coordinates) based on a probabilistic roadmap method a plurality of different postures that can be taken by the robot in the configuration space are selected as samples and the respective samples are connected in a tree structure so that the robot moves from the current posture of the robot to the robot It is possible to automatically calculate and optimize the shortest (optimal) path of the robot's moving path to the target posture that the robot takes at the final stage according to the operation of the robot, and to optimize the distribution of the samples to obtain the smallest number of samples (Optimum operation path or shortest operation path) of the robot by filling the coordinate space only with the coordinate space (cover) By increasing the computation speed, it is possible to shorten the time required to set the operation of the robot according to the user's teaching, thereby improving the real-time performance of the teaching and the responsiveness of the robot according to the teaching.

Here, the probabilistic roadmap method is an algorithm for setting an operation path of a robot while avoiding a collision between a current posture and a target posture of the robot, wherein each of the plurality of different drive axes configured in the robot is set as a coordinate axis, (Or co-ordinates on the coordinate space) in the coordinate space, and sets a plurality of different postures that the robot can take in the coordinate space in the coordinate space as the sample (or coordinate on the coordinate space) And moving the robot from the sample corresponding to the current attitude to the sample corresponding to the target attitude.

Accordingly, the robot operation modeling unit 120 can exclude the posture of the robot where the collision occurs, based on the collision attitude information provided from the control unit 130, as a posture that the robot can not take, For example, the coordinate of each collision posture according to the collision posture information in the coordinate space may be identified as an exclusion coordinate (exclusion object coordinate), and the exclusion coordinate may be set as a restricted area or a restricted space in the coordination space.

At this time, the limited area or the limited space may be composed of one or more negative coordinates.

Accordingly, the robot operation modeling unit 120 samples the coordinate space corresponding to the robot, which is preset graph information corresponding to the robot, using the probabilistic roadmap method for the remaining space excluding the limited area or the limited space in the coordinate space (randomly) a plurality of different samples corresponding to each of a plurality of different postures that the robot can take.

At this time, there has been a problem that the distribution of the samples selected in the coordination space is sampled uniformly at the time of sample selection on the coordinate space based on the probabilistic roadmap method, and is concentrated on a specific portion without being uniform.

This unnecessarily increases the number of samples to cover (fill) the coordinate space, thereby increasing the computation time required to calculate the motion path of the robot.

To solve this problem, the robot operation modeling unit 120 applies a Poisson disk sampling method for defining a sample region as a fixed disk shape among Poisson sampling methods to a sample determination method for path calculation of a robot Thereby increasing the uniformity of the density of the sample in the coordination space. Thus, it is possible to calculate the optimal path with a minimum number of samples.

For example, the robot operation modeling unit 120 may fill (cover) the entire coordinate space by randomly selecting a sample in a coordinate space, which is preset graph information according to a preset Poisson disk sampling method, It can be regarded as a hard sphere having a disk radius to have a certain area or volume.

Accordingly, the robot operation modeling unit 120 can select samples in a state in which the interval between adjacent different samples is maintained at a predetermined value or more by the radius of the disk, and the sample is not concentrated in a specific region in the coordinate space A plurality of samples can be selected so as to be uniformly distributed without reducing the number of samples required to cover the coordinate space and the time required for calculating the operation path can be significantly shortened compared with the conventional method.

At this time, the robot operation modeling unit 120 can determine any one of the number of samples and the disk radius of the sample according to the Poisson disk method according to the size of the coordinate space, The other one can be determined. In addition, the number of the samples and the disk radius may be preset in the robot operation modeling unit 120. [

3, the robot operation modeling unit 120 may form a tree structure based on the probabilistic roadmap method on samples selected in the coordinate space, and a tree structure is formed according to the selected samples And may provide information on the coordination space to the interaction detection unit 132 of the control unit 130.

Accordingly, the interaction detecting unit 132 receives information about the current attitude and the target attitude according to the teaching information recognized (generated) from the motion recognizing unit 131, or receives the teaching information, From the operation modeling unit 120, information about the coordinate space in which the individual robot positions on the coordinate space excluding the collision posture are formed in a tree structure.

Hereinafter, the interaction detecting unit 132 may identify a sample corresponding to the current posture and the target posture according to the teaching information in the coordinate space, and may identify a sample corresponding to the target posture from the sample corresponding to the current posture Can be calculated based on the probabilistic roadmap method.

If there is no sample matching the current posture or the target posture in the coordinate space, the interaction detecting unit 132 determines a sample closest to the coordinates of the coordinate space corresponding to the current posture or the target posture, And may be identified as a sample corresponding to the target posture.

That is, the interaction detecting unit 132 may calculate an optimal operation path as the shortest path for operating (moving) from the current posture of the robot according to the user's teaching to the target posture of the robot, It is possible to calculate the optimal operation path of the robot corresponding to the work operation stably and accurately taught by the user without collision by eliminating the collision posture in the coordination space in cooperation with the robot 120.

Meanwhile, the interaction detecting unit 132 may generate control information for controlling the robot in accordance with the user's instruction based on the shortest path, and may transmit the control information through the communication unit configured in the simulation apparatus 100 And transmits the control information to the robot 10 so that the control information can be set in the robot 10.

At this time, the control information may be configured as a work file for allowing the robot 10 to operate according to the shortest path generated according to the user's teaching.

Meanwhile, the image generating unit 133 applies the control information calculated corresponding to the user's teaching information to the robot model information based on the shortest path in cooperation with the interaction detecting unit 132, A virtual reality image, which is an operation image of a robot model related to a user's instruction operating according to the control information, can be generated and transmitted to the virtual reality interface device 20.

In the above-described configuration, the interaction detecting unit 132 performs a function of simulating the motion of the robot in the virtual reality in cooperation with the motion recognition unit 131, and performs all functions similar to those of a general industrial robot Is implemented in software. At this time, in the simulation of the robot operation of the interaction detecting unit 132, when the user moves the action point and programs the operation of the robot, the inverse kinematic control in the task space according to the work environment can be performed, When moving the joints directly, control in the joint space can be performed.

Also, the image generating unit 133 generates a virtual reality image by simulating the operation of the robot model for interacting with a user's operation in a virtual reality (virtual space) in real time, and transmits the generated virtual reality image to the user's virtual reality interface apparatus 20 ) To visualize the working environment of the robot and the robot in three dimensions.

For example, the interaction detecting unit 132 and the image generating unit 133 cooperate with each other to catch a point of action of the robot, move the hand in a desired direction, and generate the motion of the robot in real time through a preset simulator And visualizes it in three dimensions and provides it through the virtual reality interface device 20, whereby the operator can experience the same experience as programming the robot through direct teaching in real life.

According to the above-described configuration, the present invention realizes a working environment of a robot and a robot as a virtual reality to provide a virtual reality image, and enables a user to conveniently check the operation of the robot in real time on the virtual reality according to the virtual reality image In addition to supporting the intuitive teaching by supporting the teaching, it is also possible to automatically exclude an obstacle that is located in the working environment of the robot or the self collision of the robot, It is possible to calculate the optimal operation path of the robot in which the collision does not occur so that the operation of the robot corresponding to the stable and accurate teaching of the user can be set in the robot, .

In addition, the present invention compresses the point cloud information generated by scanning the working environment to a level that can detect whether or not the robot is in conflict with the robot during the working environment of the robot by using a hexahedron or polygon, The amount of data of the work environment model can be greatly reduced, and the load and computation time required for judging the collision of the robot in the work environment can be greatly reduced. Thus, the optimal operation path of the robot corresponding to the user's teaching is provided in real time At the same time, it is possible to guarantee immediate response of the robot model in response to the user's operation.

In addition, according to the present invention, in a sampling process of selecting, as samples, a plurality of different postures that can be taken by a robot in a coordinate space in which a posture of a robot is formed as a coordinate system based on a probabilistic roadmap method in calculating an optimal operation path, , It is possible to uniformly distribute the selected samples in the coordination space. Therefore, the optimal motion path is automatically calculated from the robot's current posture to the target posture set by the user according to the user's teaching, It is possible to improve the efficiency of the robot operation and to optimize the distribution of the sample to fill the coordinate space with the smallest number of samples to increase the operation speed for the calculation of the optimum operation path, Shorten the time required for setting, Time and the responsiveness of the robot model according to the teachings.

In the above-described configuration, the detailed configuration of the working environment modeling unit 110 will be described with reference to the drawings.

1, the work environment modeling unit 110 may include an input unit 111, a clustering unit 112, a plane determination unit 113, and a modeling unit 114.

First, the input unit 111 receives three-dimensional point-cloud information (or point-cloud data) generated by scanning a work space constituting a working environment of a robot and obstacles through the three-dimensional sensor 30, As input, and provide the corresponding point group information to the clustering unit 112.

The work environment modeling unit 110 may further include a sensor unit including the 3D sensor 30. The sensor unit may scan the work environment to generate the 3D point cloud information .

In addition, the working environment may include an obstacle occupying a part of the working space of the robot and the working space, and the obstacle may include facilities and the like.

Meanwhile, as shown in FIG. 4, the clustering unit 112 may generate a depth image based on the depth information included in the point cloud information, and may apply a predetermined image coordinate system to the depth image.

In addition, the clustering unit 112 may clusters the point cloud information in a block form on the basis of image coordinates according to an image coordinate system applied to the depth image, Can be generated.

픽셀(pixel) 단위로 구성할 수 있다.At this time, the clustering unit 112 may configure the sizes of the clusters in various ways. In general,

It can be configured in pixel units.

For example, the clustering unit 112 may generate a cluster corresponding to the block shape by clustering a plurality of different image coordinates constituting a block shape by applying a coordinate system to the depth image.

In contrast to the conventional modeling method using three-dimensional position information of points, the present invention can divide the point cloud information into clusters easily and quickly because only the image coordinates are used.

The plane determining unit 113 checks whether the plurality of different clusters are planar or not in cooperation with the clustering unit 112. If the plane is not a plane, It is possible to repeat the process of dividing and dividing the cluster until the plane is completed.

5 and 6, the planar determination unit 113 may receive the information on the clusters from the clustering unit 112. [0050]

As shown in FIG. 5, the plane determiner 113 may construct a data matrix by collecting point cloud information belonging to an image coordinate region constituting the specific cluster with respect to a specific cluster.

)으로 구성된 점군 정보를 이용하여 하기의 수학식 1과 같은 데이터 행렬(

)을 생성할 수 있다.For example, the plane determination unit 113 may determine a plurality of points (for example,

) Using the point group information composed of the data matrix < RTI ID = 0.0 >

Can be generated.

)으로 구성된 점군 정보를 이용하여 구성되며, 상기 x, y, z는 점군 정보에 따른 점의 좌표를 의미할 수 있다.As shown in the figure, the data matrix includes M points (

), And x, y, and z may be coordinates of a point according to the point cloud information.

)의 중심점(

)을 산출할 수 있다. 이때, 상기 p와

는 상호 동일할 수 있다.6 (a), the plane determiner 113 determines the data matrix (m) composed of the M points (p) through the following equation (2)

) ≪ / RTI &

) Can be calculated. At this time,

May be mutually identical.

)의 중심점(

)과 일치시키기 위하여 하기 수학식 3을 통해 상기 데이터 행렬(

)의 중심점(

)을 기준으로 상기 데이터 행렬을 구성하는 각 점(

)을 이동시켜 상기 원점과 중심점이 일치된 데이터 행렬(

)을 생성할 수 있다.6 (b), the plane determining unit 113 determines the origin of the coordinate system related to the singular value decomposition, which is set in advance for singular value decomposition,

) ≪ / RTI &

) ≪ / RTI > to the data matrix < RTI ID = 0.0 >

) ≪ / RTI &

) ≪ / RTI > of the data matrix < RTI ID = 0.0 >

) To move the origin and the center point of the data matrix (

Can be generated.

)의 중심점(

)이 원점이 되도록 상기 중심점을 기준으로 상기 데이터 행렬(

)을 이동시켜 상기 원점과 중심점이 일치된 데이터 행렬(

)을 생성할 수 있다.In other words, the plane determination unit 113 determines that the data matrix

) ≪ / RTI &

) Is the origin, the data matrix (

) To move the origin and the center point of the data matrix (

Can be generated.

)에 대하여 하기 수학식 4에 따른 미리 설정된 특이값 분해(SVD: Singular Value Decomposition)를 수행하여 상기 특정 클러스터에 대응되는 데이터 행렬의 랭크(rank)를 구할 수 있다.6 (c), the plane determiner 113 determines a data matrix (center) generated by moving the center point (center coordinate) of the data matrix corresponding to the specific cluster to the origin of the singular value decomposition related coordinate system, (

(SVD) according to Equation (4) to obtain the rank of the data matrix corresponding to the specific cluster.

로 나타낼 수 있으며, 이에 따라 상기 S는 특이값 행렬로서 변환시 가감되는 이득을 대각행렬 형태로 나타낸 것이며, U는 출력측 특이벡터로서 특이값이 나오는 출력 y의 방향벡터를 나타낸 것이고, V는 입력측 특이벡터로서 특이값이 나오는 입력 x의 방향벡터를 나타낸 것이다.At this time, in the linear transformation y = Ax,

Where S is a singular value matrix, and a gain in the diagonal matrix is represented by a diagonal matrix. U denotes a direction vector of an output y having a singular value as an output-side singular vector, and V denotes an input side singular vector And the direction vector of the input x having the singular value is shown.

For example, the plane determiner 113 may calculate the number of singular values obtained through the singular value decomposition, determine the rank of the data matrix obtained through the number of singular values, It can be determined whether or not the distribution of the points belonging to the point cloud constitutes a plane.

That is, since the rank of the data matrix indicates the dimension of the data distribution, the point cloud is distributed in the form of a straight line if the value is 1 and the point cloud is distributed in one plane if the rank is 2. If the rank is 3, .

으로 최대 랭크는 3이며, 상기 평면 판단부(113)는 상기 데이터 행렬의 랭크를 0이 아닌(0을 제외한) 특이값의 수를 세어서 파악할 수 있다.At this time, the size of the data matrix is

, The maximum rank is 3, and the plane determiner 113 can grasp the rank of the data matrix by counting the number of non-zero (exclusive of 0) singular values.

Through the above-described configuration, the plane determination unit 113 can determine whether or not a plurality of different point group information corresponding to a plurality of different clusters are distributed in one plane, It is possible to determine whether each of the other plurality of clusters is flat according to the above description.

If the point group information composed of M points belonging to a specific cluster is distributed in one plane through the above-described configuration, the specific information (for example, the plane equation and the corner information, the center coordinates of the surface, The larger the M is, the larger the compression effect on the point cloud information becomes.

Meanwhile, in the above-described configuration, the information of the three-dimensional sensor 30 that scans the work space may contain noises and the singular value may not be exactly 0. Therefore, the plane determination unit 113 may determine The value may be determined to be 0, and the threshold value may be set differently according to the accuracy of the sensor and the magnitude of the measurement noise.

)과 두 번째 특이값(

)의 비율을 확인함으로써 한 평면에 분포하는지 판별할 수 있다.Also, when there is a spatially discontinuous region in the cluster, the rank may be 2 even though the distribution of the points is not distributed in one plane, and it is necessary to check whether the points are appropriately distributed in one plane. The cluster with rank 2 is identified by the first singular value (

) And the second singular value (

), It is possible to discriminate whether or not they are distributed in one plane.

Therefore, the plane determiner 113 determines whether the parameter obtained through the singular value decomposition of the data matrix corresponding to the specific cluster satisfies Equation (5) according to the following Equation (5) It is possible to determine whether the distribution of the inner point group is made up of planes, thereby determining whether the specific cluster is flat or not.

,

,

는 크기 순서대로 표현된 데이터 행렬의 특이값이며, ε는 랭크 판별을 위한 임계치이고, μ는 평면의 가로와 세로의 비율에 대한 임계치를 각각 나타낸다.In Equation (4) and Equation (5)

,

,

Is a singular value of a data matrix expressed in magnitude order,? Is a threshold value for rank discrimination, and? Represents a threshold value for the ratio of the horizontal to vertical length of the plane, respectively.

Through the above-described configuration, the plane determining unit 113 can determine whether each of the plurality of clusters is flat or not.

If the specific cluster is found not to be planar, the plane determiner 113 may divide the specific cluster into a plurality of different sub-clusters at a preset ratio, It is possible to apply the above process of dividing the detailed clusters other than the planes as in the case of the clusters instead of the planes by applying the process for determining whether or not the clusters including the equations 1 to 5 are planar.

At this time, the plane determination unit 113 may divide the specific cluster so that a plurality of different sub-clusters divided in the specific cluster have the same size when divided into a plurality of different sub-clusters in a specific cluster.

The plane determining unit 113 provides the cluster information to the clustering unit 112 in a specific cluster determined not to be a plane, and the specific cluster, which is not a plane in the clustering unit 112, Information about each sub-cluster may be divided into sub-clusters, and the information about each sub-cluster may be provided to the planarization unit 113. The non-planar cluster may be divided into a plurality of sub-clusters in real time in cooperation with the clustering unit 112 .

In addition, the planar determination unit 113 may determine whether the detailed cluster is planar by applying the above-described configuration. If the detailed cluster is not a plane, the planar determination unit 113 may divide the detailed cluster again at a preset ratio , It is possible to determine whether the cluster is divided in the detailed cluster, and to divide the cluster again if it is not a plane.

That is, the plane determination unit 113 can repeatedly divide the cluster until a plane appears.

Accordingly, the plane determination unit 113 can repeat the division until a plane appears for a specific cluster, thereby generating a plurality of different clusters (including divided clusters) composed only of a plane.

Then, the plane determination unit 113 may provide the modeling unit 114 with cluster information for each of the clusters (including the divided clusters) formed in a plane.

In this case, the minimum size of the cluster may be set in advance by the plane determination unit 113. If the detailed cluster (or divided cluster) generated through the partitioning corresponds to the minimum size, And provides the modeling unit 114 with information on the cluster corresponding to the cluster.

Meanwhile, the modeling unit 114 may display (model) the point cloud information in a hexahedron or a polygon based on the point cloud information included in the cluster information for clusters (including the divided clusters) determined as planar.

For example, as shown in FIG. 7, the modeling unit 114 may model the point cloud information corresponding to a specific cluster in a hexahedron based on a predetermined OBB (Oriented Bounding Box) algorithm.

The OBB uses a representation of an object as a hexahedron and can quickly detect collision between different objects generated on the basis of the OBB, thereby being widely used for real-time processing such as a game.

That is, the modeling unit 114 uses the information obtained through the singular value decomposition of the data matrix corresponding to a specific cluster to obtain information on the center position, width, height, and depth of the hexahedron, Depths) of the obstacles in the work space through which the specific obstacles or the planes constituting the structure of the work space arranged in the work space constituting the working environment of the robot can be generated based on the OBB information, .

)를 이용하여 산출되며, 육면체의 넓이, 높이, 깊이는 각각 특이값

,

,

가 된다. At this time, the center position of the hexahedron is the center point information of the data matrix of Equation (2)

), And the width, height, and depth of the hexahedron are respectively calculated by using a specific value

,

,

.

]로 나타낼 수 있다.In addition, the direction vector for the width, height, and depth is a column vector of the input vector matrix V obtained by singular value decomposition, and V = [

].

Meanwhile, as shown in FIG. 8, the modeling unit 114 may model the point cloud information according to the cluster information constituting the plane as a polygon.

For example, the modeling unit 114 may model a polygon based on any one of a triangle and a quadrangle. A method of modeling a triangle may include mapping the edge information in the image coordinate among the points belonging to the cluster determined as the plane Can be stored as a corner point of a triangle.

In addition, the modeling unit 114 may project the three-dimensional information of the corner point on the plane again in the cluster to model the square, and store the information as a corner of the rectangle.

In this case, the point of the corner is projected back to the plane because the point cloud information includes noise, so that the point cloud information belonging to the plane may be distributed at random in the estimated plane, and the point related information of the corner If it is used as it is, the noise is reflected in the vertex so that it is projected again on the plane in order to remove it, so that it is converted into a point on the exact plane and utilized as edge information.

In order to do this, the modeling unit 114 first calculates the column vector of V corresponding to the third singular value in the V matrix representing the direction of the input vector obtained through the singular value decomposition as shown in Equation (6) (Center point) of the center of gravity.

Next, the modeling unit 114 obtains z by putting x, y information of a corner point of the cluster into a plane equation through the following expression (7), and obtains x, y, z as coordinates of a point projected on a plane It can be stored as rectangle corner information.

9, the modeling unit 114 can generate model information on the work environment model through modeling of the work space and the obstacle that constitute the work environment of the robot, 9 (a) is an example of a work environment model generated through an OBB, and FIG. 9 (b) is an example of a work environment model modeled based on a polygon.

The detailed configuration of the robot operation modeling unit 120 will be described below with reference to the drawings.

First, as shown in FIG. 1, the robot operation modeling unit 120 may include a sampling unit 121 and a tree generating unit 122.

The robot operation modeling unit 120 may be configured to perform a robot movement modeling based on a probabilistic roadmap method (PRM) by using a plurality of different robots that can be taken by a robot in a configuration space configured by a coordinate system, The robot is moved from the current position of the robot to the target position taken by the robot according to the operation of the robot according to the user's teaching, (Optimal) path for the moving motion path can be automatically calculated and simulated, and the distribution of the sample can be optimized to fill the coordinate space with the smallest number of samples (cover) By increasing the computation speed for the computation of the path, the time required for the operation setting of the robot according to the user's teaching is shortened Support to improve the responsiveness of the robot according to the real-time and its teachings than teaching.

At this time, the probabilistic roadmap method is an algorithm for setting an operation path of a robot while avoiding a collision between a current posture and a target posture of the robot, wherein each of the plurality of different drive axes configured in the robot is set as a coordinate axis, (Or co-ordinates on the coordinate space) in the coordinate space, and sets a plurality of different postures that the robot can take in the coordinate space in the coordinate space as the sample (or coordinate on the coordinate space) And moving the robot from the sample corresponding to the current attitude to the sample corresponding to the target attitude.

The sampling unit 121 samples a coordinate space corresponding to the robot and is preset graph information by using a probabilistic roadmap method so that a plurality of different The sample can be selected arbitrarily (randomly).

In this case, as shown in FIG. 10 (a), a coordinate space is uniformly sampled when a sample is selected on a coordinate space based on a probabilistic roadmap method, so that distribution of samples selected in the coordinate space is not uniform There was a problem in that it was concentrated on the part.

This unnecessarily increases the number of samples to cover (fill) the coordinate space, thereby increasing the computation time required to calculate the motion path of the robot.

In order to solve this problem, the sampling unit 121 applies a Poisson disk sampling method for defining a sample region as a fixed disk shape among Poisson sampling methods to a sample determination method for path calculation of a robot, By increasing the uniformity of the sample density of the space, it is possible to calculate the optimal path with a minimum number of samples.

For example, as shown in FIG. 11, the sampling unit 121 may randomly select a sample in a coordinate space, which is preset graph information according to a preset Poisson disk sampling method, and the selected sample has a predetermined area or volume It can be regarded as a hard sphere having a disk radius.

10 (b), the sampling unit 121 can select samples in a state in which the interval between adjacent different samples is maintained at a certain level or more by the radius of the disk, It is possible to select a plurality of samples so that the sample is uniformly distributed without being concentrated in a specific region and at the same time, the number of samples required to cover the coordinate space can be reduced, .

At this time, the sampling unit 121 determines one of the number of the samples and the disk radius of the sample according to the Poisson disk method according to the size of the coordinate space, You can decide the other one. Also, at least one of the number of the samples and the radius of the disc may be set in the sampling unit 121 in advance.

In one example, the sampling unit 121 may determine the number of the samples and calculate the disk radius required to fill the coordinate space according to the determined number of samples.

At this time, the sampling unit 121 may determine the number of samples to be selected in the coordinate space according to Equation (8).

는 최적 구의 패킹 밀도(optimal sphere packing density),

는 디스크 반경(poisson disk sampling 반경), n은 상기 배위공간의 차원, Γ(x)는 감마함수, vol(Q)는 상기 배위 공간의 부피일 수 있다.At this time, N is the number of sample nodes (the number of samples)

Is the optimal sphere packing density,

(X) is a gamma function, and vol (Q) is a volume of the coordinate space.

Also, the sampling unit 121 may determine the disk radius of the selected sample through the Poisson disk method using the following equation (9).

는 최적 구의 패킹 밀도(optimal sphere packing density),

는 디스크 반경(poisson disk sampling 반경), n은 상기 배위공간의 차원, Γ(x)는 감마함수, vol(Q)는 상기 배위 공간의 부피일 수 있다.At this time, N is the number of sample nodes (the number of samples)

Is the optimal sphere packing density,

(X) is a gamma function, and vol (Q) is a volume of the coordinate space.

Accordingly, the sampling unit 121 can repeat the above-described sampling process to select a plurality of samples for the entire coordinate space.

As described above, the sampling unit 121 may cover the entire coordinate space with a smaller number of samples than the sampling according to the conventional uniform method by applying the Poisson sampling algorithm in the sampling process according to the probabilistic roadmap method. It is possible to find out the robot's motion path more optimized than the conventional one.

12 (a), the tree generating unit 122 generates a tree between adjacent samples among a plurality of different samples selected in the coordinate space in cooperation with the sampling unit 121, And can form a tree structure by interconnecting them according to a method.

At this time, as described above, the sampling unit receives the collision attitude information from the interaction detecting unit 132, and controls the attitude of the obstacle disposed in the working environment or the attitude that the robot can not take (For example, a posture in which a collision with an obstacle occurs, or a posture in which the robot itself collides with an internal structure of the robot) can be set.

Accordingly, the sampling unit 121 can sample a plurality of different samples as described above with respect to the rest of the coordinate space excluding the limited region or the limited space. In addition, the tree generating unit 122 may connect the selected samples through the sampling unit 121 according to the probabilistic roadmap method to form a tree.

12 (a), the interaction detecting unit 132 included in the control unit 130 may operate in conjunction with at least one of the sampling unit 121 and the tree generating unit 122, A plurality of different samples corresponding to the current posture A and the target posture B according to the teaching information among the plurality of distributed samples can be identified.

Here, the teaching information may include attitude information about the current posture and the target posture of the robot, the current posture of the robot means a start posture at which the robot starts to move, It may mean an attitude that operates from the starting posture and ends the motion of the robot.

Also, the interaction detecting unit 132 may detect a plurality of different samples corresponding to the current posture A and the target posture B using the tree structure based on a plurality of samples connected in a tree structure in the coordination space The shortest path (the shortest operation path or the optimum operation path) between the samples can be calculated.

According to the above-described configuration, the present invention can provide a shortest path for moving the robot from the current posture A to the target posture B according to the user's teaching as shown in Fig. 12 (a) 12 (b), it is possible to greatly improve the operation efficiency of the robot by excluding the inefficient operation path calculated according to the conventional sampling method, and to minimize the number of samples required for calculation of the shortest operation path, The load required for calculation and the calculation time can be greatly shortened.

Accordingly, it is an object of the present invention to reduce the load and computation time required for calculating an efficient operation path of the robot, and to promptly change the operation of the robot in real time, So that the real-time performance and the direct reactivity to the robot operation setting can be effectively improved.

13 is a simulation method for teaching a virtual reality-based robot of a simulation apparatus 100 for teaching a virtual reality-based robot in association with a user's virtual reality interface apparatus 20 for teaching a robot according to an embodiment of the present invention FIG.

First, the simulation apparatus 100 receives the point cloud information generated by scanning the working environment through the three-dimensional sensor 30, clusters based on the coordinates of the depth image obtained based on the depth information according to the point cloud information The planar cluster can be modeled as a polygon or a hexahedron to generate environment model information for a reduced-capacity work environment (S1).

In addition, the simulation apparatus 100 may be configured to perform a simulation on the basis of the environment model information and the robot model information in a predetermined coordinate space in which the individual posture of the robot is set to coordinates, Poisson disk) sampling method to form a tree structure for calculating the operation path of the robot (S2).

Next, the simulation apparatus 100 generates an image of the virtual reality interface based on the environment model information and the preset robot model information about the robot, and provides the image through the virtual reality interface apparatus 20 (S3) , And receives teaching information including the user's current posture and the desired posture for robot teaching from the virtual reality interface device (S4).

Thereafter, the simulation apparatus 100 generates a shortest path for moving the robot from the current posture to the target posture based on the teaching information and the tree structure, and reflects the shortest path to the robot model information It is possible to provide one image (S5).

The various devices and components described herein may be implemented by hardware circuitry (e.g., CMOS-based logic circuitry), firmware, software, or a combination thereof. For example, it can be implemented utilizing transistors, logic gates, and electronic circuits in the form of various electrical structures.

It will be apparent to those skilled in the art that various modifications and variations can be made in the present invention without departing from the spirit or essential characteristics thereof. Therefore, the embodiments disclosed in the present invention are intended to illustrate rather than limit the scope of the present invention, and the scope of the technical idea of the present invention is not limited by these embodiments. The scope of protection of the present invention should be construed according to the following claims, and all technical ideas within the scope of equivalents should be construed as falling within the scope of the present invention.

100: simulation apparatus 110: work environment modeling unit
111: input unit 112: clustering unit
113: plane determination unit 114: modeling unit
120: Robot motion modeling unit 121: Sampling unit
122: tree generating unit 130:
131: motion recognition unit 132: interaction detection unit
133:

Claims (10)

1. A simulation apparatus for teaching a virtual reality-based robot interlocked with a user's virtual reality interface apparatus for teaching a robot,
The point cloud information is clustered on the basis of the coordinates of the depth image obtained on the basis of the depth information according to the point cloud information to form a plane cluster as a polygon A work environment modeling unit for modeling the work environment model to generate environment model information for a work environment reduced in capacity;
A robot operation modeling unit that forms a tree structure for calculating the operation path of the robot by filling a preset coordinate space in which the individual postures of the robot are set in coordinates with arbitrary samples by a Poisson disk sampling method; And
A virtual reality interface unit for generating an image of a virtual reality interface based on the environment model information and preset robot model information about the robot and providing the virtual reality interface through the virtual reality interface unit, And a robot controller for receiving the teaching information including the robot model information and the robot model information in cooperation with the robot modeling unit and the working environment modeling unit, And a controller for calculating an optimal path for moving the robot from the current posture to the target posture based on the tree structure and reflecting the optimal path to the robot model information to provide a generated image,
Based virtual reality robot.
The method according to claim 1,
Wherein the work environment modeling unit verifies whether each of the clusters is planar, and classifies clusters that are not planar, until a plane is confirmed.
The method according to claim 1,
The work environment modeling unit
An input unit for receiving the point cloud information generated by scanning the working environment of the robot;
A clustering unit for clustering the point cloud information in a block form based on coordinates of a depth image obtained based on depth information according to the point cloud information;
A planar determination unit configured to construct a data matrix of a point group belonging to an image coordinate region of each cluster, determine whether the planar region is a plane through the rank of the data matrix, determine whether the planar region is planar, And
A modeling unit for modeling the clusters determined by the plane as a plane polygon or a hexahedron to reduce the amount of data
Wherein the virtual reality-based robot teaching system comprises:
The method according to claim 1,
Wherein the control unit maps the individual posture of the robot at the time of generating the optimal path and generates the robot model information corresponding to the optimal path so as to interact with the user's operation based on the preset robot model information modeled by the robot And transmits the generated motion image to the virtual reality interface device.
The method according to claim 1,
The control unit detects collision between the robot and the working environment or whether the robot itself collides with the individual posture of the robot based on the environment model information and the robot model information, collects the collision posture in which the collision is detected, Sets a restricted area or a limited space corresponding to the collision posture in the space,
Wherein the robot operation modeling unit fills the arbitrary sample with respect to the rest space excluding the limited area or the limited space in the coordinate space.
The method according to claim 1,
Wherein the controller calculates the optimal path based on a preset probabilistic roadmap method (PRM) in the optimal path calculation.
The method according to claim 1,
The robot motion modeling unit
A sampling unit for selecting the arbitrary sample arbitrarily selected in the coordinate space by the Poisson disk sampling method and repeating the sampling process to select a sample for the entire coordinate space; And
A tree generating unit for forming a tree structure based on the selected sample,
Wherein the virtual reality-based robot teaching system comprises:
The method according to claim 1,
Wherein the control unit generates control information for controlling the operation of the robot on the basis of the optimum path corresponding to the user's teaching.
The method of claim 8,
And a communication unit for connection with the robot,
Wherein the control unit transmits the control information to the robot through the communication unit so that a work operation according to the optimum path corresponding to the control information is set in the robot.
1. A simulation method for teaching a virtual reality-based robot of a simulation apparatus interlocked with a user's virtual reality interface apparatus for teaching a robot,
The point cloud information is clustered on the basis of the coordinates of the depth image obtained on the basis of the depth information according to the point cloud information to form a plane cluster as a polygon Generating environmental model information for a work environment having a reduced capacity by modeling the work environment with a hexahedron;
A Poisson disk sampling method is used to randomly sample the rest of the robot except for the posture of the robot where collision occurs based on the environment model information and the robot model information in a predetermined coordinate space in which the individual posture of the robot is set to coordinates Forming a tree structure for calculating an operation path of the robot;
A virtual reality interface unit for generating an image of a virtual reality interface based on the environment model information and preset robot model information about the robot and providing the virtual reality interface through the virtual reality interface unit, And a target posture; And
Generating an optimal path for moving the robot from the current posture to a target posture based on the teaching information and the tree structure, and providing an image generated by reflecting the optimal path to the robot model information
A method for simulating virtual reality based robot teaching.

Images