KR102277162B1 - Apparatus for Monitoring Industrial Robot and Driving Method Thereof - Google Patents

Abstract

The present invention relates to an industrial robot monitoring apparatus and a driving method of the apparatus. The industrial robot monitoring apparatus according to an embodiment of the present invention comprises: a communication interface unit installed at an industrial site to receive a photographed image of a working industrial robot; and a control unit that analyzes the received image to calculate a motion result for a skeleton of the industrial robot, compares the calculated movement result with the skeleton data, which is the reference data related to the movement of the skeleton, and determines whether the industrial robot is operating abnormally based on the comparison result. The present invention can detect defects at an early stage.

Description

Industrial robot monitoring device and driving method thereof {Apparatus for Monitoring Industrial Robot and Driving Method Thereof}

The present invention relates to an industrial robot monitoring device and a driving method thereof, and more particularly, in the case of an industrial robot, for example, it is usually configured in an arm (ARM) shape, and a plurality of joints are configured according to the industrial purpose to optimize the process It is manufactured to have movement, and it relates to an industrial robot monitoring apparatus and a driving method of the apparatus that allow higher malfunction detection by paying attention to this specific form.

In general, an industrial robot is a device that realizes factory automation as a facility that works on behalf of humans, such as not only large industrial facilities such as heavy industry and aviation industry, but also small industrial facilities such as semiconductor industry, manufacturing industry process systems, and the like. Modern manufacturing industrial process systems are large and complex, and many processes are more dependent on industrial robots than on human workers, and interest in maintaining reliability and stability of industrial robot systems in automatic processes is increasing. In addition, as high-speed mass production is possible through industrial robots, many products can be produced in a short time. However, if a problem occurs in the industrial robot, the equipment cannot be operated and production must be stopped for a certain period of time, so damage caused by equipment non-operation has a big problem.

In order to solve this problem, there is a technique known in the prior art for installing a camera in the field and monitoring the robot work quality through the camera captured image.

However, since the prior art detects and uses motion from an image, the number of times of detection of a malfunction is too high and the accuracy thereof is low, there is still a problem that the production process must be stopped from time to time. In this regard, there is an urgent need for a method to accurately detect a malfunction.

Korean Patent Publication No. 10-1487169 (2015.01.22) Korean Patent Publication No. 10-2018-0019705 (2018.02.26) Korean Patent Publication No. 10-2017-0083256 (2017.07.18) Korean Patent Publication No. 10-2019-0104934 (2019.09.11) Korean Patent Publication No. 10-2020-0132768 (2020.11.25)

In the embodiment of the present invention, for example, an industrial robot is usually configured in the form of an arm (ARM), and a plurality of joints are configured according to the degree of industry to have optimal movement in the process. An object of the present invention is to provide an industrial robot monitoring device for sensing and a driving method of the device.

Industrial robot monitoring apparatus according to an embodiment of the present invention, a communication interface unit for receiving a photographed image of an industrial robot installed at an industrial site and working, and movement of the industrial robot with respect to the skeleton by analyzing the received photographed image and a controller that calculates a result, compares the calculated motion result with skeleton data, which is reference data related to the motion of the skeleton, and determines whether the industrial robot operates abnormally based on the comparison result.

The control unit may determine whether the movements of the skeleton at any time point tn coincide with each other in a cycle in which the same operation is repeated.

The controller may compare a movement distance on the X, Y, and Z axes of the skeleton and a rotation angle on each axis.

The controller may further determine the state of the object transferred by the industrial robot in order to determine the abnormal operation of the industrial robot.

The controller may accumulate and store the calculated motion results, and predict whether the robot is abnormal by analyzing the stored accumulated data by applying artificial intelligence (AI).

As a result of the analysis of the accumulated data, the controller may predict whether the error occurs differently depending on the time change within an error range of a normal operation or continuously occurs for a specified period.

The industrial robot monitoring apparatus further includes a storage unit for storing first skeleton data for a first industrial robot and second skeleton data for a second industrial robot installed in different industrial sites, wherein the control unit includes the industrial robot Based on the identification information of the robot, the stored first skeleton data or the second skeleton data may be detected and selectively compared.

In addition, the driving method of the industrial robot monitoring apparatus according to an embodiment of the present invention comprises the steps of: a communication interface unit receiving a photographed image of an industrial robot installed in an industrial site and working; and a control unit analyzing the received photographed image to calculate the motion result for the skeleton of the industrial robot, compare the calculated motion result with the skeleton data, which is reference data related to the motion of the skeleton, and determine whether the industrial robot operates abnormally based on the comparison result includes

In the determining step, it may be determined whether the movements of the skeleton at any time point tn coincide with each other in a cycle in which the same operation is repeated.

In the determining step, the movement distance on the X, Y and Z axes of the skeleton and the rotation angle on each axis may be compared.

The determining may further determine a state of an object transported by the industrial robot in order to determine an abnormal operation of the industrial robot.

In the driving method, the calculated motion result is accumulated and stored, and the stored accumulated data is analyzed by applying artificial intelligence (AI) to predict whether the robot is abnormal.

The predicting may include predicting the abnormality when an error occurs differently depending on the time change within an error range of a normal operation as a result of analyzing the accumulated data or continuously occurs for a specified period.

The driving method further comprises the step of storing the first skeleton data for the first industrial robot and the second skeleton data for the second industrial robot that the storage unit is installed in different industrial sites, the determining step is the Based on the identification information of the industrial robot, the stored first skeleton data or the second skeleton data may be detected and selectively compared.

According to an embodiment of the present invention, it is possible to detect an early defective product by detecting abnormal movement of a robot in a high-tech process such as a semiconductor or a display. In other words, it will be possible to reduce the increase in material cost (cost) through early detection of defects in the process and disposal of the product.

In addition, according to an embodiment of the present invention, abnormal prediction monitoring may be performed using motion data.

1 is a view showing an industrial robot monitoring system according to an embodiment of the present invention;
Figure 2 is a diagram showing the monitoring state of the industrial robot of Figure 1,
3 is a diagram illustrating the operation cycle process of the industrial robot;
4 is a view illustrating the configuration of the industrial robot of FIG. 1;
5 is a block diagram illustrating a detailed structure of the industrial robot monitoring apparatus of FIG. 1;
6 is a view showing a monitoring process of an industrial robot according to an embodiment of the present invention, and
7 is a flowchart illustrating a driving process of the industrial robot monitoring apparatus of FIG. 1 .

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

1 is a diagram showing an industrial robot monitoring system according to an embodiment of the present invention, FIG. 2 is a diagram schematically showing the monitoring state of the industrial robot of FIG. 1, FIG. 3 is a diagram illustrating the operation cycle process of the industrial robot, and 4 is a diagram illustrating the configuration of the industrial robot of FIG. 1 .

As shown in FIG. 1 , the industrial robot monitoring system 90 according to an embodiment of the present invention includes some or all of the photographing device 100 , the communication network 110 , and the industrial robot monitoring device 120 .

Here, "including some or all" means that some components such as the communication network 110 are omitted so that the imaging device 100 and the industrial robot monitoring device 120 perform direct communication (eg, P2P communication), or It means that some or all of the components constituting the industrial robot monitoring device 120 can be configured by being integrated into a network device (eg, a wireless exchange device, etc.) constituting the communication network 110 , In order to facilitate a sufficient understanding, it is explained that everything is included.

Prior to the detailed description, briefly looking at the industrial robot 97, the industrial robot 97 is frequently used in high-tech processes such as semiconductors or displays, or a welding robot widely used for welding of automobile bodies is also used as the industrial robot 97. ) are included. There is a spot welding robot as a welding robot, and since spot welding is done in a point form, it is also called spot welding, and unlike rivet welding in which a hole is directly drilled and joined to a plate, it can be joined without drilling a hole. Of course, the industrial robot 97 is widely used in industrial sites, and is manufactured and used in various forms according to the characteristics of the industrial site. For example, the number of joints of the industrial robot 97 may be different.

2 to 4 show an industrial robot 97 . As shown in FIG. 2 , the industrial robot 97 moves in the X, Y, and Z-axis directions with respect to the floor surface, and each axis may rotate at the same time. Of course, here, the rotation can be expressed as an angle. It can be seen that the industrial robot 97 is capable of 6-axis driving in a three-dimensional space. Of course, such a robot is composed of a body 400 and an arm 410 connected to the body 400 as shown in FIG. 4 . Of course, here the body 400 and the arm 410, and the arm 410 and the arm 410 are connected by a joint.

In addition, as can be seen in FIGS. 2 to 4 , the industrial robot 97 of FIG. 1 may operate in the same manner as in FIG. 3 according to characteristics of an industrial site. As shown in FIG. 4 , the arm 410 is moved in a designated manner to operate in the order of 1 to 5 as shown in FIG. 3 . In other words, the industrial robot 97 repeats one cycle process that operates in the order of 1 to 5. For example, consider a robot that lifts and moves wafers on a semiconductor production line. The suction plate attached to the arm of the robot moves downward to suck the wafer. Then, the arm is moved to move the adsorbed wafer to the next process, and the moved wafer is separated. That is, the adsorption is canceled. The robot repeats this process. Therefore, the operation of adsorbing the wafer or glass plate to the suction plate, lifting the wafer and moving the wafer to the next process, and then removing the waiter from the suction plate can be viewed as a circulation process. Figure 3 can be seen to show just such a cycle process. In the one-cycle process tn, operation 1 may be a time point (or point) t1, and 2 may be a time point t2.

The photographing device 100 photographs the industrial robot 97 that is installed and operated in an actual industrial site. The industrial robot 97 is a robot when the body 400 is fixed and operated as shown in FIG. 4, but there may also be a robot that moves along a specific rail, such as a transport vehicle, and the photographing device 100 is such an industrial robot. The movement of (97) is photographed. In an embodiment of the present invention, the photographing apparatus 100 may be installed inside an industrial site to perform a photographing operation. Of course, it is desirable to photograph the industrial robot 97 up close.

The photographing apparatus 100 may use various types of cameras. Although CCTV may be used, an IP camera may be used, and a three-dimensional camera may also be used. In the case of a 3D camera, since it is possible to obtain metadata in 3D space, it may be possible to directly generate and provide coordinate values for the movement of the industrial robot 97 . In other words, data related to the X, Y, Z axes, and movement angles for each axis may be directly generated based on the floor surface of the industrial site where the industrial robot 97 is installed and provided as metadata. Accordingly, since meta data can be secured in various forms in an embodiment of the present invention, it will not be particularly limited to any one form. Above all, the photographing device 100 provides a photographed image of the industrial robot 97 . In addition, in the case of a 3D camera, metadata may be provided together with a corresponding captured image. Since the meta data may include various data, it will not be particularly limited to any one.

In addition, the photographing device 100 may be installed in the vicinity and interwork with an edge device that primarily analyzes the photographed image of the photographing apparatus 100 . As we enter the era of the 4th industrial revolution (eg, autonomous vehicles, big data processing, etc.), the amount of data processing rapidly increases, which can be seen to reduce the computational burden caused by data processing and to reduce the overload in the communication network 110 . have. Therefore, in the case of the edge device, it is possible to periodically update the program related to the monitoring of the industrial robot 97 according to the instruction of the industrial robot monitoring device 120, for example, by updating a preset program. . Typically, the edge device provides the analysis result of the captured image, but when there is a request from the industrial robot monitoring device 120, only video frames of a specific time period may be acquired or extracted and provided. Of course, the video frame request of the specific time period may be used when an error of the industrial robot 97 is detected and more accurate analysis is required. In this way, the photographing apparatus 100 may operate in various forms, such as further using an edge device.

The communication network 110 may include an intranet, which is an in-house communication network, and includes all of various types of wired and wireless communication networks. For example, a wired/wireless Internet network may be used or interlocked as the communication network 110 . Here, the wired network includes an Internet network such as a cable network or a public telephone network (PSTN), and the wireless communication network includes CDMA, WCDMA, GSM, Evolved Packet Core (EPC), Long Term Evolution (LTE), Wibro network, etc. is meant to include Of course, the communication network 110 according to the embodiment of the present invention is not limited thereto, and may be used, for example, in a cloud computing network under a cloud computing environment, a 5G network, and the like. For example, when the communication network 110 is a wired communication network, the access point in the communication network 110 can connect to a switching center of a telephone company. or by accessing various repeaters such as a Base Transmissive Station (BTS), NodeB, and e-NodeB to process data.

The communication network 110 may include an access point (AP). Here, the access point includes a small base station such as a femto or pico base station that is often installed in a building. Femto or pico base stations are classified according to the maximum number of access to the imaging device 100, etc. in the classification of small base stations. Of course, the access point may include a short-distance communication module for performing short-distance communication such as Zigbee and Wi-Fi with the photographing device 100 and the like. The access point may use TCP/IP or Real-Time Streaming Protocol (RTSP) for wireless communication. Here, short-range communication may be performed in various standards such as Bluetooth, Zigbee, infrared, radio frequency (RF) such as ultra high frequency (UHF) and very high frequency (VHF), and ultra wideband communication (UWB) in addition to Wi-Fi. Accordingly, the access point can extract the location of the data packet, specify the best communication path for the extracted location, and forward the data packet to the next device, for example, the industrial robot monitoring device 120 along the designated communication path. The access point may share several lines in a general network environment, and includes, for example, a router, a repeater, and a repeater.

The industrial robot monitoring device 120 may include a server provided for each industrial site, and may include a platform server for integrated monitoring of robots used in all industrial sites. The industrial robot monitoring device 120 may store motion data for the industrial robots 97 installed at each industrial site, more precisely, skeleton data on the movement of the skeleton as reference data, in the DB 120a. For example, the skeleton data as the reference data may be initially installed in an industrial site to acquire, store, and use data during normal operation of the corresponding industrial robot 97 . Alternatively, by referring to a data sheet provided by a manufacturer for manufacturing the industrial robot 97, the corresponding data may be acquired and stored. Since data can be acquired and stored through various paths, the embodiment of the present invention will not be limited to any one form. For example, if the data on the data sheet is used, but it does not exactly match after the industrial robot 97 is installed at the industrial site, but is within the error range of normal operation, the error can be regarded as a characteristic of the industrial site and can be secured as reference data. .

The industrial robot monitoring device 120 builds, for example, movement data related to the movement of the industrial robot 97 installed in the actual industrial site, more precisely, the movement of the skeleton in the DB 120a of FIG. 1 , and then photographing the industrial site. A robot object may be extracted from the captured image received from the device 100 , and the movement of the skeleton of the extracted robot object may be tracked, and a robot malfunction may be detected by comparing the tracking result with pre-stored skeleton data. Of course, it is clear that the comparison should be made with respect to the operation at the same time point. In other words, if the time point in the third operation was t3 when the industrial robot 97 operates with the operations 1 to 5 in a cycle cycle as in FIG. 3 above, the motion tracking result obtained by analyzing the captured image is also in the cycle cycle. It is necessary to compare the results of the operation when the third operation of t3 corresponds to each other. For example, if they do not coincide with each other, it may be determined as an abnormal operation, that is, a malfunction of the industrial robot 97 . Of course, the industrial robot monitoring device 120 does not make a final judgment as a malfunction immediately as a single oil level is found, and of course it can be determined immediately, but when it exceeds the reference value after continuous monitoring, it is finally determined as a malfunction and takes appropriate action make it happen

Above all, it can be considered more important that the industrial robot monitoring device 120 in the embodiment of the present invention accurately predicts the abnormal operation of the industrial robot 97 through the analysis of the captured image. For example, a motion tracking result for the movement of the industrial robot 97 is stored according to time change, and of course, such data can be viewed as data about the movement of a skeleton, which is accumulated and stored according to time change and then artificial It will be possible to accurately predict abnormal operation due to aging of the industrial robot 97 using deep learning of intelligence. Typically, compared to the reference data of the initial normal operation, that is, compared with the skeleton data, the abnormal operation is not abnormal, but when an error continuously occurs for several days or several hours, it is finally determined to be abnormal and measures can be taken in advance. For example, if it is within the error range of normal operation, it is not determined that the product is defective.

For example, the industrial robot monitoring apparatus 120 may further utilize various data even when determining such abnormal operation. For example, when the industrial robot 97 moves an object from a specific location to another location, the degree or location of the shaking of the object may be checked. Therefore, when the position of the finally moved object reaches the level of mass production of defective products, abnormal operation can be finally determined. In addition, it is possible to determine the year of manufacture, such as the year of manufacture of the industrial robot 97 , or use data about the industrial robot 97 in another field to see what abnormalities are found in relation to the corresponding industrial robot 97 . In this way, the abnormal operation is finally determined by comprehensively utilizing various types of data.

In summary, the industrial robot monitoring device 120 extracts the powder, that is, the skeleton from the photographed image of the normal operating robot, sets the normal operating range in the powdered state, and uses it for a specific operating time (eg, order) By extracting each image frame, length (or movement distance) and angle data on the X, Y, and Z axes can be analyzed. Since the movement of the robot received from one camera moves in three directions, the length and angle of the skeleton for X, Y, and Z change according to the operation time (order). Therefore, the length and angle of the skeleton of each image frame are stored in a predetermined time period (in units of several seconds or the sequence of stop operation, etc.) up to one cycle process, which is the standard for normal operation. Then, the industrial robot monitoring device 120 receives the image taken by the working robot during the actual process work, extracts the skeleton in a specific time period from the working robot image, and lengths for X, Y, and Z of the skeleton extracted from the robot image during the operation And angle is extracted, and the length and angle for X, Y, and Z of the skeleton of the robot during operation are compared with the skeleton for analysis in the same time period. In this case, the above-stored skeleton data for analysis may be used. According to the comparison result, the industrial robot monitoring device 120 determines normal operation and malfunction.

Above all, the industrial robot monitoring device 120 according to the embodiment of the present invention detects the movement of the powder, that is, the skeleton of the industrial robot 97 in order to increase the accuracy of detecting a malfunction, rather than detecting the entire motion of the industrial robot 97. . Therefore, when the industrial robot 97 is composed of a plurality of skeletons, it is possible to detect whether a malfunction is detected by tracking the movement of the skeleton, which is the center of the plurality of skeletons, or is considered the most important. In addition, under certain conditions, the accuracy of judging the malfunction of the industrial robot 97 can be increased by extending the motion tracking to the surrounding skeleton or increasing the number of skeletons tracking the motion gradually (or stepwise). . In other words, as the number of skeletons used to determine a malfunction increases, the objectivity of the data is guaranteed, so the detailed method can be designed and utilized in various ways according to the intention of the system designer. For example, it is good to use motion tracking for many skeletons from the beginning, but in this case, the data processing load increases and the data processing speed decreases accordingly. , and predicts whether the industrial robot 97 malfunctions based on the tracking result, further confirms the movement of the surrounding skeleton when a malfunction is detected according to the prediction, or increases the number of skeletons to be monitored according to time change. When an error occurs even in the skeleton, it can be finally determined as a malfunction. In addition, the industrial robot monitoring device 120 may operate in the form of returning to the initial operation if it is not a big problem as a result of determining whether a malfunction occurs while increasing the number of surrounding skeletons or the skeleton.

As a result of the above configuration, the embodiment of the present invention may detect abnormal movements of the robot to detect early defective products, and abnormal predictive monitoring using motion data may be possible. Furthermore, it will be possible to save material costs due to product disposal according to the rapid detection of anomaly prediction.

5 is a block diagram illustrating a detailed structure of the industrial robot monitoring apparatus of FIG. 1 .

As shown in FIG. 5 , the industrial robot monitoring apparatus 120 of FIG. 1 according to an embodiment of the present invention includes a communication interface unit 500 , a control unit 510 , a skeleton determining unit 520 and a storage unit 530 . includes some or all of

The communication interface unit 500 communicates with the photographing apparatus 100 via the communication network 110 of FIG. 1 . When the photographing device 100 interworks with the edge device, the communication interface 500 may communicate with the edge device. In addition, in the process of performing communication, the communication interface unit 500 may perform various operations such as modulation/demodulation, muxing/demuxing, encoding/decoding, etc., which are obvious to those skilled in the art, and thus further description will be omitted.

The communication interface unit 500 transmits the photographed image received from the photographing device 100 , that is, the photographed image of the industrial robot 97 to the controller 510 . Also, when there is a control request from the photographing apparatus 100 , the communication interface unit 500 may transmit a control signal to the photographing apparatus 100 to adjust the photographing angle. Of course, the photographing apparatus 100 uses a fixed camera, but a PTZ (Pan, Tilt, Zoom) camera may be used, and when a PTZ camera is used, control may be performed in various forms. However, it is preferable that the control of the camera is controlled when the photographing device 100 is initially installed at the industrial site. For example, when the control of the photographing apparatus 100 occurs due to unavoidable circumstances, the movement of the industrial robot 97, more Precisely, an operation for setting a reference for the movement of the skeleton may be performed again.

The control unit 510 is responsible for the overall control operation of the communication interface unit 500, the skeleton determining unit 520, and the storage unit 530 of FIG. 5 constituting the industrial robot monitoring device 120 of FIG. 1 . For example, when a captured image is provided from the communication interface unit 500 , it may be temporarily stored in the storage unit 530 . In addition, when the industrial robot monitoring device 120 of FIG. 1 monitors different types of industrial robots 97 installed in multiple industrial sites, the corresponding captured images are stored together with identification information of the corresponding industrial robots 97 . After being stored in the unit 530, abnormal operation determination may be made using different reference data, that is, skeleton data. To this end, the control unit 510 interworks with the skeleton determination unit 520 .

In other words, the control unit 510 may execute a program mounted on the skeleton determination unit 520 in order to determine the malfunction of the industrial robot 97 according to the embodiment of the present invention. In addition, the control unit 510 may perform various operations according to the determination result of the skeleton determination unit 520 , for example, the analysis result of the captured image is generated in a specified format and systematically stored in the DB 120a of FIG. 1 . It is classified and stored, and when abnormal operation is determined, the fact is notified to the manager terminal device such as a smartphone or computer of the manager who manages the industrial robot 97 according to the request of the skeleton determination unit 520. will be able to give

The skeleton determining unit 520 generates movement data, that is, skeleton data, on the movement of various types of industrial robots 97 installed in the industrial site, more precisely, the movement of the skeleton, and the DB 120a of FIG. 1 or the storage unit ( 530) can be stored. Alternatively, it can be stored in the registry inside the program. For example, the industrial robot 97 may have various shapes depending on which industrial site is provided and operated, and a plurality of skeletons, that is, each arm or powder, may be configured based on the body of the industrial robot 97. have. For example, suppose there are two powders. In this case, the skeleton determining unit 520 secures movement data for the skeleton during normal operation, for example, through analysis of a captured image, or by using a data sheet provided by the manufacturer of the industrial robot 97. . In this case, as for the skeleton data, which is movement data, movement distances of the X, Y, and Z axes and rotation angles in each axis may be secured with respect to the floor surface of the industrial site.

The skeleton determining unit 520 analyzes the photographed image of the current operation of the industrial robot 97 received at the industrial site in a state in which the skeleton data is built, and compares it with the skeleton data previously stored as reference data. Of course, in the case of comparison, comparison can be made for each skeleton, that is, for each powder. In other words, when the industrial robot 97 has a skeleton A based on the body, and a skeleton A and a skeleton B are connected through a joint, the movement of the skeleton A and the skeleton B may be compared with reference data, respectively. Alternatively, skeleton A and skeleton B can be compared together. Of course, each of these comparison operations may be performed, but since it is possible to do it comprehensively, the embodiment of the present invention will not be particularly limited to any one form.

In addition, the skeleton determining unit 520 may determine the characteristics of the object transported by the industrial robot 97 in order to more accurately determine the malfunction of the industrial robot 97 . Here, the characteristic of the object may include a movement position and movement of the object. Through this, the abnormal operation of the robot can be determined more comprehensively. In addition, age information of the industrial robot 97 may be utilized, or data on the operating state of the same type of industrial robot 97 used in other industrial sites may be utilized. For example, when there are many errors in the industrial robot 97 of a specific year, a malfunction may be predicted based on the corresponding information.

Furthermore, the skeleton determining unit 520 generates big data by collecting motion data, that is, movement data of the skeleton according to time change of the industrial robot 97, and predicts abnormal behavior by applying a program such as artificial intelligence deep learning. can The industrial robot 97 may be continuously monitored when, for example, 10 years have elapsed as aging progresses. Accordingly, when an error occurs within the error range, although it is not a malfunction, it can be checked whether the error continues to occur. For example, when an error occurs over several days, a malfunction is predicted in this case, so that the manager can be notified in advance. In this way, since various data can be additionally used to finally determine the abnormal operation of the industrial robot 97, the embodiment of the present invention will not be particularly limited to any one form.

As described above, the skeleton determining unit 520 is capable of monitoring to determine the error between the numerical value of the robot operation program and the actual physical movement, and notifies the abnormality of the movement in real time, so that early detection of defective products is possible, and the accumulated It is possible to predict and monitor robot abnormalities using motion identification data. In addition, the skeleton determining unit 520 can also grasp the position and movement of the product when the robot is manipulated. Of course, this monitoring operation is performed through analysis of the captured image, and the reference at this time may be the floor or the initial posture of the body.

The storage unit 530 may temporarily store various data processed under the control of the control unit 510 . For example, the storage unit 530 may temporarily store the photographed image of the photographing apparatus 100 of FIG. 1 provided from the control unit 510 , and then call it and provide it to the skeleton determination unit 520 . In addition, when the storage unit 530 provides the analysis result of the captured image provided by the skeleton determining unit 520 in a specified format, it temporarily stores it in the storage unit 530 and then the DB of FIG. 1 under the control of the control unit 510 . (120a) may be provided.

In addition to the above, the communication interface unit 500, the control unit 510, the skeleton determining unit 520, and the storage unit 530 of FIG. 5 may perform various operations, and other details have been sufficiently described above, so the contents I want to replace them with

Meanwhile, as another embodiment of the present invention, the control unit 510 may include a CPU and a memory, and may be formed as a single chip. The CPU includes a control circuit, an arithmetic unit (ALU), an instruction interpreter and a registry, and the memory may include a RAM. The control circuit performs a control operation, the operation unit performs an operation operation of binary bit information, and the instruction interpretation unit converts a high-level language into a machine language and a machine language into a high-level language, including an interpreter or compiler. , the registry may be involved in software data storage. According to the above configuration, for example, at the beginning of the operation of the industrial robot monitoring device 120, the program stored in the skeleton determining unit 520 is copied and loaded into a memory, that is, RAM, and then executed to increase the data operation processing speed. can be increased quickly.

6 is a diagram illustrating a monitoring process of an industrial robot according to an embodiment of the present invention.

Referring to FIG. 6 together with FIG. 1 for convenience of explanation, the industrial robot monitoring apparatus 120 of FIG. 1 may execute, for example, a robot operation program to generate and pre-store movement result data for commands as metadata (S600, S610).

For example, the data on the movement of the industrial robot 97 installed in the industrial site, more precisely, the movement of the skeleton, is generated by analyzing the photographed image provided when the photographing device 100 is initially installed in the industrial site, or , may be generated using data of a data sheet provided by the manufacturer of the industrial robot 97 . Of course, since the characteristics of each industrial site may be somewhat different, it is possible to pre-store the analysis result as metadata by analyzing the captured image and checking whether it is within the range of the data sheet based on the data in the data sheet. Of course, the metadata here may include data on the movement of the industrial robot 97 during one cycle of the process cycle, more precisely, the movement of the skeleton.

In addition, the industrial robot monitoring device 120 stores the received captured image in the memory and then performs image analysis, extracts the industrial robot in the video frame as an object in the analysis process, and can track the movement of the extracted object ( S611 to S614).

Through this process, metadata including motion data of an object related to an industrial robot in an image may be generated (S615). For example, if the compact metadata is the first metadata, the metadata obtained by photographing the industrial robot 97 that is actually working may be called the second metadata.

The industrial robot monitoring device 120 compares the first meta data that is inconsistent with the second meta data acquired through the industrial robot 97 that is actually working ( S616 ). Of course, the comparison here may be a comparison of the movement of the skeleton, for example, the movement distances of the X-axis, the Y-axis, and the Z-axis may be compared, and the rotation angle in each axis may be compared.

The industrial robot monitoring device 120 repeatedly performs a comparison operation according to time change, and determines an abnormal situation of the industrial robot 97 based on the result (S617). For example, since the industrial robot 97 is capable of 6-axis driving by the control of a motor, etc., the industrial robot monitoring device 120 determines that the operation is normal when the data are compared and match each other or are within an error range. However, when they do not match each other or are out of the error range, they are judged as abnormal. Of course, in order to determine the abnormal operation, various operations may be performed, and since this has been sufficiently described above, the contents will be substituted.

In addition, the industrial robot monitoring device 120 collects the metadata of the industrial robot 97 operating in real time and analyzes the collected data (S618, S619).

The industrial robot monitoring device 120 may predict an abnormal situation of the industrial robot 97 by analyzing the accumulated data, for example, through artificial intelligence (S620). As described above, it is not a defect, but an error is continuously occurring, and if the age of the industrial robot 97 is old, compare it with case data of the same industrial robot 97 in other industrial sites, and an abnormal situation that may come predictable. It can be seen that there is a difference in AI in that prediction is possible from the existing rule-based data analysis method, and since various types of prediction are possible, the embodiment of the present invention is not particularly limited to any one type. won't

7 is a flowchart illustrating a driving process of the industrial robot monitoring apparatus of FIG. 1 .

Referring to FIG. 7 together with FIG. 1 for convenience of explanation, the industrial robot monitoring apparatus 120 according to an embodiment of the present invention receives a photographed image of an industrial robot being installed and working at an industrial site ( S700 ).

Prior to this, the industrial robot monitoring apparatus 120 may store, as reference data, the skeleton data for the skeleton movement of the industrial robot in the three-dimensional space. Here, the skeleton data may include data such as movement distances on the X-axis, Y-axis, and Z-axis for the powder of the industrial robot 97 , that is, the skeleton, and the rotation angle of each axis. For example, if a specific skeleton of the industrial robot 97 in one cycle process is at movement distances of a, b, and c with respect to the X-axis, Y-axis, and Z-axis, respectively, and the rotation angle on the X-axis is d, this It stores the data for reference as reference data, that is, as skeleton data.

In addition, the industrial robot monitoring device 120 analyzes the received captured image to calculate the motion result for the skeleton of the industrial robot, compares the calculated motion result with the skeleton data, which is reference data related to the motion of the skeleton, and compares the result. It is determined whether the industrial robot operates abnormally based on the basis (S710). By extracting an object for the industrial robot 97 from the photographed image of the industrial robot 97 currently working, and comparing the motion tracking and tracking results of the extracted object with pre-stored skeleton data, it is determined whether the results are consistent with each other and abnormal judge the action.

For example, the industrial robot monitoring device 120 receives data of a photographed image of a working robot during actual process work, and extracts a skeleton of a specific time period, that is, a specified time point from the received working robot image. That is, it can be said that the skeleton is extracted from the video frame at the corresponding time point. Then, the length on the X, Y, and Z axes of the skeleton extracted from the robot image during operation, that is, the movement distance and angle are extracted. Then, by comparing the pre-stored skeleton data with the length and angle on the X, Y, and Z axes of the skeleton of the robot you are working with, and the skeleton for analysis in the same time period, it is possible to determine whether it is normal or abnormal.

In addition to the above, the industrial robot monitoring apparatus 120 of FIG. 1 according to an embodiment of the present invention can perform various operations, and other detailed information has been sufficiently described above, so it will be replaced with the contents.

On the other hand, even though it has been described that all components constituting the embodiment of the present invention are combined or operated as one, the present invention is not necessarily limited to this embodiment. That is, within the scope of the object of the present invention, all the components may operate by selectively combining one or more. In addition, although all of the components may be implemented as one independent hardware, some or all of the components are selectively combined to perform some or all functions of the combined components in one or a plurality of hardware program modules It may be implemented as a computer program having Codes and code segments constituting the computer program can be easily deduced by those skilled in the art of the present invention. Such a computer program is stored in a computer-readable non-transitory computer readable media, read and executed by the computer, thereby implementing an embodiment of the present invention.

Here, the non-transitory readable recording medium refers to a medium that stores data semi-permanently and can be read by a device, not a medium that stores data for a short moment, such as a register, cache, memory, etc. . Specifically, the above-described programs may be provided by being stored in a non-transitory readable recording medium such as a CD, DVD, hard disk, Blu-ray disk, USB, memory card, ROM, and the like.

In the above, preferred embodiments of the present invention have been illustrated and described, but the present invention is not limited to the specific embodiments described above, and it is common in the technical field to which the present invention pertains without departing from the gist of the present invention as claimed in the claims. Various modifications may be made by those having the knowledge of, of course, and these modifications should not be individually understood from the technical spirit or perspective of the present invention.

97: industrial robot 100: imaging device
110: communication network 120: industrial robot monitoring device
400: (robot) body 410: arm (or skeleton, powder)
500: communication interface unit 510: control unit
520: skeleton determination unit 530: storage unit

Claims (8)

A communication interface unit installed at an industrial site to receive a photographed image of an industrial robot working; and
Analyze the received captured image to calculate the motion result for the skeleton of the industrial robot, compare the calculated motion result with the skeleton data, which is reference data related to the motion of the skeleton, based on the comparison result of the industrial robot Including; a control unit for determining whether an abnormal operation
The control unit accumulates and stores the calculated motion results, and predicts whether the robot is abnormal by analyzing the stored accumulated data by applying artificial intelligence (AI),
The control unit, as a result of the analysis of the accumulated data, an industrial robot monitoring device for predicting the abnormality when an error occurs differently depending on the time change within an error range of normal operation or continuously occurs for a specified period. According to claim 1,
The control unit, an industrial robot monitoring device for determining whether the movement of the skeleton at a certain time point (tn) in a cycle in which the same operation is repeated is consistent with each other. According to claim 1,
The control unit, an industrial robot monitoring device for comparing the movement distance on the X, Y and Z axes of the skeleton and the rotation angle on each axis. According to claim 1,
The control unit, industrial robot monitoring device for further determining the state of the object transported by the industrial robot in order to determine the abnormal operation of the industrial robot. delete delete According to claim 1,
It further includes; a storage unit for storing the first skeleton data for the first industrial robot and the second skeleton data for the second industrial robot installed in different industrial sites;
The control unit, an industrial robot monitoring device for selectively comparing the first skeleton data or the second skeleton data stored on the basis of the identification information of the industrial robot to detect. Receiving, by a communication interface unit, a photographed image of an industrial robot installed in an industrial site and working; and
A control unit analyzes the received captured image to calculate a motion result for the skeleton of the industrial robot, compares the calculated motion result with reference data related to the motion of the skeleton, and compares the result with the skeleton data based on the comparison result Determining whether the industrial robot is operating abnormally; including,
predicting, by the control unit, whether the robot is abnormal by accumulating and storing the calculated motion results, and analyzing the stored accumulated data by applying artificial intelligence (AI); and
Predicting whether the error occurs when the error occurs differently according to the time change within the error range of the normal operation as a result of the analysis of the accumulated data or continues to occur for a specified period, by the control unit;
Driving method of the industrial robot monitoring device further comprising.

Images