CN114378835B - Robot control system based on image recognition and control method thereof - Google Patents
Robot control system based on image recognition and control method thereof Download PDFInfo
- Publication number
- CN114378835B CN114378835B CN202210299746.1A CN202210299746A CN114378835B CN 114378835 B CN114378835 B CN 114378835B CN 202210299746 A CN202210299746 A CN 202210299746A CN 114378835 B CN114378835 B CN 114378835B
- Authority
- CN
- China
- Prior art keywords
- analysis
- monitoring
- robot
- adjustment
- robot hand
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
- B25J9/1602—Programme controls characterised by the control system, structure, architecture
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J19/00—Accessories fitted to manipulators, e.g. for monitoring, for viewing; Safety devices combined with or specially adapted for use in connection with manipulators
- B25J19/0095—Means or methods for testing manipulators
Landscapes
- Engineering & Computer Science (AREA)
- Robotics (AREA)
- Mechanical Engineering (AREA)
- Automation & Control Theory (AREA)
- Manipulator (AREA)
Abstract
The invention discloses a robot control system based on image recognition and a control method thereof, which relate to the technical field of robot control and solve the technical problem that the operation state of a robot cannot be accurately judged by the image recognition technology in the operation process in the prior art, monitor the operation process of the currently used robot, monitor the operation of the current robot by the image recognition technology and judge whether the operation of the current robot is normal or not, thereby improving the intelligent control of the robot, simultaneously ensuring the operation efficiency of the robot and preventing the reduction of the production efficiency caused by the abnormal operation of the robot; the robot hand with abnormal operation monitoring is subjected to feature extraction, the analysis accuracy of the robot hand fault is improved, the control of the robot hand operation is enhanced by collecting the fault features of the robot hand, the fault risk of the robot hand is effectively reduced, the working efficiency of the robot hand is maximized, and the robot hand with abnormal operation monitoring is marked as an abnormal analysis object.
Description
Technical Field
The invention relates to the technical field of robot control, in particular to a robot control system based on image recognition and a control method thereof.
Background
The robot control system has advanced with the development of a robot. The manipulator is developed on the basis of an ancient robot appearing in the early period, the research on the manipulator starts in the middle of the 20 th century, and along with the development of computers and automation technology, particularly since the first digital electronic computer appeared in 1946, the computer has made remarkable progress and is developed towards the direction of high speed, large capacity and low price. Meanwhile, the urgent need of mass production promotes the development of automation technology, and lays a foundation for the development of robots and manipulator control systems. On the other hand, the research of nuclear energy technology requires some handling machinery to handle radioactive materials instead of humans.
However, in the prior art, the robot hand cannot accurately judge the running state of the robot hand through an image recognition technology in the running process, so that the robot hand cannot be accurately controlled according to the real-time state, and in addition, the robot hand cannot be reasonably controlled according to the accurate characteristics of equipment in the control process of the robot hand, so that the control efficiency and accuracy of the robot hand are reduced, and meanwhile, the production progress is influenced, and the working efficiency is reduced.
In view of the above technical drawbacks, a solution is proposed.
Disclosure of Invention
The invention aims to solve the problems, and provides a robot control system and a control method thereof based on image recognition, which are used for monitoring the operation process of a currently used robot, monitoring the operation of the current robot through the image recognition technology and judging whether the operation of the current robot is normal or not, thereby improving the intelligent control of the robot, simultaneously ensuring the operation efficiency of the robot and preventing the reduction of the production efficiency caused by the abnormal operation of the robot; the robot hand with abnormal operation monitoring is subjected to feature extraction, the analysis accuracy of the robot hand fault is improved, the control of the robot hand operation is enhanced by collecting the fault features of the robot hand, the fault risk of the robot hand is effectively reduced, the working efficiency of the robot hand is maximized, and the robot hand with abnormal operation monitoring is marked as an abnormal analysis object.
The purpose of the invention can be realized by the following technical scheme:
a robot control system based on image recognition comprises a server, wherein the server is in communication connection with an operation process monitoring unit, a quality prediction unit, an intelligent control unit, a feature extraction unit and an action adjustment monitoring unit;
the server generates an operation process monitoring signal and sends the operation process monitoring signal to the operation process monitoring unit, the operation process monitoring unit monitors the operation process of the currently used robot, and if the operation monitoring of the corresponding robot is abnormal, a feature extraction signal is generated and sent to the feature extraction unit; the feature extraction unit is used for extracting features of the robot with abnormal operation monitoring, generating qualified feature data and non-qualified feature data through the feature extraction, and sending the qualified feature data and the non-qualified feature data to the intelligent control unit; if the operation monitoring of the corresponding manipulator is normal, generating an operation monitoring normal signal and sending the operation monitoring normal signal to a server;
after receiving the qualified characteristic data and the non-qualified characteristic data, the intelligent control unit intelligently controls the current robot; the motion adjusting and monitoring unit monitors the intelligent control of the robot hand in the operation process; and after the monitoring and analysis are completed, the operation quality of the robot hand is predicted through the quality prediction unit.
As a preferred embodiment of the present invention, the operation monitoring process is as follows:
setting a monitoring time period, dividing the monitoring time period into i sub-time points, monitoring the robot hand in the monitoring time period, acquiring the moving part of the robot hand through monitoring, carrying out image interception by taking each sub-time point as a node, marking an image corresponding to each sub-time point as an analysis picture, and then sequencing the analysis pictures according to the arrangement sequence of each sub-time point; randomly selecting movable parts of the robot hand, marking the movable parts as analysis parts, periodically dividing analysis pictures according to each position of the analysis parts in an operation cycle, and marking the analysis pictures in the same period as an analysis picture group; comparing the analysis pictures after the period division is completed in real time, acquiring a position difference value of an analysis part corresponding to the analysis pictures in the analysis picture groups of the adjacent periods and a speed difference value of the corresponding analysis part in operation by taking the analysis picture groups of the adjacent periods as comparison objects, and comparing the position difference value of the analysis part corresponding to the analysis pictures in the analysis picture groups of the adjacent periods and the speed difference value of the corresponding analysis part in operation with a position difference value threshold value and a speed difference value threshold value respectively:
if the position difference value of the analysis part corresponding to the analysis picture in the adjacent period analysis picture group exceeds a position difference value threshold value or the speed difference value of the operation of the corresponding analysis part exceeds a speed difference value threshold value, judging that the operation monitoring of the corresponding robot is abnormal, generating a feature extraction signal and sending the feature extraction signal to a feature extraction unit; if the position difference value of the analysis part corresponding to the analysis picture in the adjacent period analysis graph group does not exceed the position difference value threshold value and the speed difference value of the operation of the corresponding analysis part does not exceed the speed difference value threshold value, judging that the operation monitoring of the corresponding robot hand is normal, generating an operation monitoring normal signal and sending the operation monitoring normal signal to the server.
As a preferred embodiment of the present invention, the process of feature extraction is as follows:
the method comprises the steps that a robot hand with abnormal operation monitoring is marked as an abnormal analysis object, analysis graph groups of the abnormal analysis object are collected, at least two analysis graph groups exist in a monitoring time period, and if feature extraction signals are generated in the comparison process of the analysis graph groups of adjacent periods, the corresponding analysis graph groups of the adjacent periods are uniformly marked as abnormal graph groups; on the contrary, if the analysis graph groups of the adjacent periods generate normal operation monitoring signals in the comparison process, the analysis graph groups corresponding to the adjacent periods are uniformly marked as normal graph groups; respectively selecting the same analysis picture of the same analysis part in the period from the normal picture group and the abnormal picture group for comparison, and extracting part data of the analysis pictures of the normal picture group and the abnormal picture group, wherein the part data comprises the angle change value and the movement speed of the selected analysis part in the corresponding analysis pictures; if the data of the internal bits of the analyzed pictures corresponding to the normal graph group and the abnormal graph group are consistent, marking the data of the corresponding parts as qualified characteristic data; if the data of the internal bits of the analysis pictures corresponding to the normal graph group and the abnormal graph group are inconsistent, marking the data of the corresponding parts as non-qualified characteristic data; and sending the qualified feature data and the non-qualified feature data to the intelligent control unit together.
In a preferred embodiment of the present invention, the monitoring process of the motion adjustment monitoring unit is as follows:
monitoring the operation of the robot hand, marking the robot hand with action adjustment as an adjustment analysis object, setting a mark k as a natural number more than 1, acquiring the frequency of the adjustment analysis object for action adjustment and the interval duration for action adjustment, and marking the frequency of the adjustment analysis object for action adjustment and the interval duration for action adjustment as PLk and SCk respectively; acquiring the frequency of the adjustment analysis object for repeatedly performing the same action adjustment, and marking the frequency of the adjustment analysis object for repeatedly performing the same action adjustment as TZk; analyzing and acquiring an action adjustment monitoring coefficient Xk of an adjustment analysis object, and comparing the action adjustment monitoring coefficient Xk of the adjustment analysis object with an action adjustment monitoring coefficient threshold value: if the action adjustment monitoring coefficient Xk of the adjustment analysis object exceeds the action adjustment monitoring coefficient threshold value, judging that the action adjustment monitoring of the corresponding adjustment analysis object is unqualified, generating an action adjustment abnormal signal and sending the action adjustment abnormal signal to an intelligent control unit, generating a pause operation instruction and sending the pause operation instruction to a server after the intelligent control unit receives the action adjustment abnormal signal, and performing shutdown maintenance on the corresponding manipulator by the server; and if the action adjustment monitoring coefficient Xk of the adjustment analysis object does not exceed the action adjustment monitoring coefficient threshold, judging that the action adjustment monitoring of the corresponding adjustment analysis object is qualified, generating an action adjustment normal signal and sending the action adjustment normal signal to the quality prediction unit.
As a preferred embodiment of the present invention, the quality prediction process is as follows:
collecting the product clamping qualification rate of the robot in the operation process and the repeated process frequency of the robot in the operation process, and respectively marking the product clamping qualification rate of the robot in the operation process and the repeated process frequency of the robot in the operation process as J and L; analyzing the quality prediction coefficient Y of the robot hand by analyzing and acquiring the quality prediction coefficient Y of the robot hand, and if the quality prediction coefficient Y of the robot hand is in an ascending trend, predicting the quality of the robot hand to be in a low-quality trend; and if the quality prediction coefficient Y of the robot is in a descending trend, predicting the quality of the robot to be in a high quality trend.
Compared with the prior art, the invention has the beneficial effects that:
1. in the invention, the operation process of the current used robot hand is monitored, the current operation of the robot hand is monitored through an image recognition technology, and whether the current operation of the robot hand is normal or not is judged, so that the intelligent control of the robot hand is improved, the operation efficiency of the robot hand can be ensured, and the reduction of the production efficiency caused by the abnormal operation of the robot hand is prevented; the robot hand with abnormal operation monitoring is subjected to feature extraction, the analysis accuracy of the robot hand fault is improved, the control of the robot hand operation is enhanced by collecting the fault features of the robot hand, the fault risk of the robot hand is effectively reduced, the working efficiency of the robot hand is maximized, and the robot hand with abnormal operation monitoring is marked as an abnormal analysis object.
2. According to the invention, the intelligent control of the robot hand in the operation process is monitored, and whether the action adjustment of the robot hand is qualified is judged, so that whether the intelligent control of the robot hand is normal is judged, the robot hand is ensured to be adjusted in time after deviation occurs in the operation process, the operation efficiency of the robot hand is ensured, the production progress can be ensured, and the influence caused by faults is reduced to the minimum; the operation quality of the robot hand is predicted according to the operation state of the robot hand, so that the probability of avoiding the fault risk of the robot hand is enhanced, and meanwhile, the influence caused by equipment fault in the production process is reduced to the minimum.
Drawings
In order to facilitate understanding for those skilled in the art, the present invention will be further described with reference to the accompanying drawings.
Fig. 1 is a schematic block diagram of the present invention.
Detailed Description
The technical solutions of the present invention will be described clearly and completely with reference to the following embodiments, and it should be understood that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Referring to fig. 1, a robot control system based on image recognition includes a server, the server is connected with an operation process monitoring unit, a quality prediction unit and an intelligent control unit in a communication manner, and the intelligent control unit is connected with a feature extraction unit and an action adjustment monitoring unit in a communication manner;
in the prior art, the robot hand is used as an important tool for industrial production, plays a positive role in promoting industrial generation, and is mainly a system for controlling the robot hand; the server is used as the core of the robot control system and used for carrying out data signal transmission on the operation control of the robot, and when the robot is put into use, the server generates an operation process monitoring signal and sends the operation process monitoring signal to the operation process monitoring unit; after operation process monitoring unit received operation process monitoring signal, carry out operation process monitoring to the robot that puts into use at present, monitor the operation of current robot through image recognition technology, judge whether the operation of current robot is normal to improve the intelligent control to the robot, can guarantee the operating efficiency of robot simultaneously, prevent that the robot operation from unusually leading to production efficiency to reduce, concrete operation process monitoring process is as follows:
setting a monitoring time period, dividing the monitoring time period into i sub-time points, monitoring the robot hand in the monitoring time period, and acquiring the moving part of the robot hand through monitoring, wherein the moving part of the robot hand is represented as a part of the robot hand changing the spatial position along with the operation in the operation process; image interception is carried out by taking each sub time point as a node, an image corresponding to each sub time point is marked as an analysis picture, and then the analysis pictures are sequenced according to the arrangement sequence of each sub time point;
the method comprises the following steps of selecting a movable part of a manipulator at will, and marking the movable part as an analysis part, wherein in the application, if the movable part of the manipulator is only one, the corresponding movable part is selected, and if not, any movable part can be selected; periodically dividing the analysis pictures through analyzing each position of the part in the operation period, and marking the analysis pictures in the same period as an analysis picture group; comparing the analysis pictures after the period division is completed in real time, acquiring a position difference value of an analysis part corresponding to the analysis pictures in the analysis picture group of the adjacent period and a speed difference value of the corresponding analysis part by taking the analysis picture group of the adjacent period as a comparison object, and comparing the position difference value of the analysis part corresponding to the analysis pictures in the analysis picture group of the adjacent period and the speed difference value of the corresponding analysis part with a position difference value threshold value and a speed difference value threshold value respectively: if the position difference value of the analysis part corresponding to the analysis picture in the adjacent period analysis graph group exceeds a position difference value threshold value or the speed difference value of the operation of the corresponding analysis part exceeds a speed difference value threshold value, judging that the operation monitoring of the corresponding robot is abnormal, generating a feature extraction signal and sending the feature extraction signal to a feature extraction unit; if the position difference value of the analysis part corresponding to the analysis picture in the adjacent period analysis graph group does not exceed the position difference value threshold value and the speed difference value of the operation of the corresponding analysis part does not exceed the speed difference value threshold value, judging that the operation monitoring of the corresponding robot hand is normal, generating an operation monitoring normal signal and sending the operation monitoring normal signal to the server;
after receiving the feature extraction signal, the feature extraction unit extracts features of the robot hand with abnormal operation monitoring, improves the analysis accuracy of the fault of the robot hand, enhances the control on the operation of the robot hand by acquiring the fault features of the robot hand, effectively reduces the fault risk of the robot hand, maximizes the working efficiency of the robot hand, marks the robot hand with abnormal operation monitoring as an abnormal analysis object, acquires an analysis chart group of the abnormal analysis object, has at least two analysis chart groups in a monitoring time period, and uniformly marks the corresponding adjacent period analysis chart groups as the abnormal chart groups if the analysis chart groups of the adjacent periods generate the feature extraction signal in the comparison process; on the contrary, if the analysis graph groups of the adjacent periods generate normal operation monitoring signals in the comparison process, the analysis graph groups corresponding to the adjacent periods are uniformly marked as normal graph groups;
respectively selecting the same analysis picture of the same analysis part in the period from the normal picture group and the abnormal picture group for comparison, and extracting part data of the analysis pictures of the normal picture group and the abnormal picture group, wherein the part data comprises the angle change value and the movement speed of the selected analysis part in the corresponding analysis pictures; if the data of the internal bits of the analyzed pictures corresponding to the normal graph group and the abnormal graph group are consistent, marking the data of the corresponding parts as qualified characteristic data; if the data of the internal bits of the analysis pictures corresponding to the normal graph group and the abnormal graph group are inconsistent, marking the data of the corresponding parts as non-qualified characteristic data; sending the qualified feature data and the non-qualified feature data to an intelligent control unit;
after the intelligent control unit receives the qualified feature data and the non-qualified feature data, the intelligent control unit intelligently controls the current robot, controls the robot according to the qualified feature data and the non-qualified feature data, adjusts the non-qualified feature data of the corresponding analysis part of the robot, and simultaneously generates an action adjustment monitoring signal and sends the action adjustment monitoring signal to the action adjustment monitoring unit;
the action adjustment monitoring unit monitors intelligent control of the robot in the operation process after receiving the action adjustment monitoring signal, the action adjustment is represented by the adjustment of converting non-qualified characteristic data of the robot into qualified characteristic data, and whether the action adjustment of the robot is qualified is judged, so that whether the intelligent control of the robot is normal is judged, the robot is timely adjusted after deviation occurs in the operation process, the operation efficiency of the robot is ensured, the production progress can be ensured, and the influence caused by faults is reduced to the minimum;
monitoring the operation of the robot hand, marking the robot hand with action adjustment as an adjustment analysis object, setting a mark k as a natural number more than 1, acquiring the frequency of the adjustment analysis object for action adjustment and the interval duration for action adjustment, and marking the frequency of the adjustment analysis object for action adjustment and the interval duration for action adjustment as PLk and SCk respectively; acquiring the frequency of the adjustment and analysis object for repeatedly performing the same action adjustment, and marking the frequency of the adjustment and analysis object for repeatedly performing the same action adjustment as TZk;
by the formula
Acquiring an action adjustment monitoring coefficient Xk of an adjustment analysis object, wherein a1, a2 and a3 are preset proportionality coefficients, a1 is more than a2 is more than a3 is more than 0, and beta is an error correction factor and takes the value of 1.36; adjusting the motion of the analysis object by the monitoring coefficient Xk and motionAnd (3) adjusting a monitoring coefficient threshold value for comparison:
if the action adjustment monitoring coefficient Xk of the adjustment analysis object exceeds the action adjustment monitoring coefficient threshold value, judging that the action adjustment monitoring of the corresponding adjustment analysis object is unqualified, generating an action adjustment abnormal signal and sending the action adjustment abnormal signal to an intelligent control unit, generating a pause operation instruction and sending the pause operation instruction to a server after the intelligent control unit receives the action adjustment abnormal signal, and performing shutdown maintenance on the corresponding manipulator by the server; if the action adjustment monitoring coefficient Xk of the adjustment analysis object does not exceed the action adjustment monitoring coefficient threshold, judging that the action adjustment monitoring of the corresponding adjustment analysis object is qualified, generating an action adjustment normal signal and sending the action adjustment normal signal to a quality prediction unit;
after receiving the operation monitoring normal signal and the action adjusting normal signal, the server generates a quality prediction signal and sends the quality prediction signal to a quality prediction unit, the quality prediction unit is used for predicting the operation quality of the robot hand and predicting the operation quality of the robot hand according to the operation state of the robot hand, so that the probability of avoiding the fault risk of the robot hand is enhanced, and meanwhile, the influence caused by equipment faults in the production process is reduced to the minimum, and the specific quality prediction process is as follows:
collecting the product clamping qualification rate of the robot in the operation process and the repeated process frequency of the robot in the operation process, and respectively marking the product clamping qualification rate of the robot in the operation process and the repeated process frequency of the robot in the operation process as J and L; by the formula
Acquiring a quality prediction coefficient Y of the robot hand, wherein e is a natural constant, alpha is an action adjustment coefficient, alpha is 2 when action adjustment exists, and alpha is 0.5 when action adjustment does not exist; analyzing the quality prediction coefficient Y of the robot hand, and if the quality prediction coefficient Y of the robot hand is in an ascending trend, predicting the quality of the robot hand to be in a low-quality trend; and if the quality prediction coefficient Y of the robot is in a descending trend, predicting the quality of the robot to be in a high quality trend.
A robot control method based on image recognition comprises the following specific steps:
the method comprises the following steps of firstly, monitoring the operation process of a currently-used robot, monitoring the operation of the current robot through an image recognition technology, and judging whether the operation of the current robot is normal or not, so that the intelligent control of the robot is improved, the operation efficiency of the robot can be ensured, and the reduction of the production efficiency caused by the abnormal operation of the robot is prevented;
step two, feature extraction, namely performing feature extraction on the robot with abnormal operation monitoring, and generating qualified feature data and non-qualified feature data through the feature extraction; the method has the advantages that the analysis accuracy of the fault of the robot hand is improved, the control on the operation of the robot hand is enhanced by collecting the fault characteristics of the robot hand, and the fault risk of the robot hand is effectively reduced, so that the working efficiency of the robot hand is maximized;
step three, intelligent control, namely intelligently controlling the current robot, controlling according to qualified feature data and unqualified feature data, and adjusting the unqualified feature data of the part of the robot corresponding to the analysis;
step four, adjustment monitoring, namely monitoring the intelligent control of the robot in the operation process and judging whether the action adjustment of the robot is qualified or not so as to judge whether the intelligent control of the robot is normal or not;
and step five, predicting the quality, namely predicting the operation quality of the robot hand and predicting the operation quality of the robot hand according to the operation state of the robot hand.
The formulas are obtained by acquiring a large amount of data and performing software simulation, and the coefficients in the formulas are set by the technicians in the field according to actual conditions;
when the robot is used, the operation process monitoring unit is used for monitoring the operation process of the currently used robot, and if the operation monitoring of the corresponding robot is abnormal, a feature extraction signal is generated and sent to the feature extraction unit; the characteristic extraction unit is used for extracting the characteristics of the robot with abnormal operation monitoring, generating qualified characteristic data and non-qualified characteristic data through characteristic extraction, and sending the qualified characteristic data and the non-qualified characteristic data to the intelligent control unit; if the operation monitoring of the corresponding robot hand is normal, generating an operation monitoring normal signal and sending the operation monitoring normal signal to a server; the intelligent control unit receives the qualified characteristic data and the non-qualified characteristic data and then intelligently controls the current robot hand; the motion adjusting and monitoring unit monitors the intelligent control of the robot in the operation process; and after the monitoring and analysis are completed, the operation quality of the robot hand is predicted through the quality prediction unit.
The preferred embodiments of the invention disclosed above are intended to be illustrative only. The preferred embodiments are not intended to be exhaustive or to limit the invention to the precise forms disclosed. Obviously, many modifications and variations are possible in light of the above teaching. The embodiments were chosen and described in order to best explain the principles of the invention and the practical application, to thereby enable others skilled in the art to best utilize the invention. The invention is limited only by the claims and their full scope and equivalents.
Claims (4)
1. A robot control system based on image recognition is characterized by comprising a server, wherein the server is in communication connection with an operation process monitoring unit, a quality prediction unit, an intelligent control unit, a feature extraction unit and an action adjustment monitoring unit;
the server generates an operation process monitoring signal and sends the operation process monitoring signal to the operation process monitoring unit, the operation process monitoring unit monitors the operation process of the currently used robot, and if the operation monitoring of the corresponding robot is abnormal, a feature extraction signal is generated and sent to the feature extraction unit; the characteristic extraction unit is used for extracting the characteristics of the robot with abnormal operation monitoring, generating qualified characteristic data and non-qualified characteristic data through characteristic extraction, and sending the qualified characteristic data and the non-qualified characteristic data to the intelligent control unit; if the operation monitoring of the corresponding robot hand is normal, generating an operation monitoring normal signal and sending the operation monitoring normal signal to a server;
after receiving the qualified characteristic data and the non-qualified characteristic data, the intelligent control unit intelligently controls the current robot; the motion adjusting and monitoring unit monitors the intelligent control of the robot in the operation process; after monitoring and analysis are completed, the operation quality of the robot hand is predicted through a quality prediction unit;
the operation process monitoring process comprises the following steps:
setting a monitoring time period, dividing the monitoring time period into i sub-time points, monitoring the robot hand in the monitoring time period, acquiring the moving part of the robot hand through monitoring, carrying out image interception by taking each sub-time point as a node, marking an image corresponding to each sub-time point as an analysis picture, and then sequencing the analysis pictures according to the arrangement sequence of each sub-time point; randomly selecting the moving part of the robot hand, marking the moving part as an analysis part, periodically dividing analysis pictures through each position of the analysis part in an operation period, and marking the analysis pictures in the same period as an analysis picture group; comparing the analysis pictures after the period division is completed in real time, acquiring a position difference value of an analysis part corresponding to the analysis pictures in the analysis picture group of the adjacent period and a speed difference value of the corresponding analysis part by taking the analysis picture group of the adjacent period as a comparison object, and comparing the position difference value of the analysis part corresponding to the analysis pictures in the analysis picture group of the adjacent period and the speed difference value of the corresponding analysis part with a position difference value threshold value and a speed difference value threshold value respectively:
if the position difference value of the analysis part corresponding to the analysis picture in the adjacent period analysis graph group exceeds a position difference value threshold value or the speed difference value of the operation of the corresponding analysis part exceeds a speed difference value threshold value, judging that the operation monitoring of the corresponding robot is abnormal, generating a feature extraction signal and sending the feature extraction signal to a feature extraction unit; if the position difference value of the analysis part corresponding to the analysis picture in the adjacent period analysis graph group does not exceed the position difference value threshold value and the speed difference value of the operation of the corresponding analysis part does not exceed the speed difference value threshold value, judging that the operation monitoring of the corresponding robot hand is normal, generating an operation monitoring normal signal and sending the operation monitoring normal signal to the server;
the process of feature extraction is as follows:
the method comprises the steps that a robot hand with abnormal operation monitoring is marked as an abnormal analysis object, analysis graph groups of the abnormal analysis object are collected, at least two analysis graph groups exist in a monitoring time period, and if feature extraction signals are generated in the comparison process of the analysis graph groups of adjacent periods, the corresponding analysis graph groups of the adjacent periods are uniformly marked as abnormal graph groups; on the contrary, if the analysis graph groups of the adjacent periods generate normal operation monitoring signals in the comparison process, the analysis graph groups corresponding to the adjacent periods are uniformly marked as normal graph groups; respectively selecting the same analysis picture of the same analysis part in the period from the normal picture group and the abnormal picture group for comparison, and extracting part data of the analysis pictures of the normal picture group and the abnormal picture group, wherein the part data comprises the angle change value and the movement speed of the selected analysis part in the corresponding analysis pictures; if the data of the internal bits of the analyzed pictures corresponding to the normal graph group and the abnormal graph group are consistent, marking the data of the corresponding parts as qualified characteristic data; if the data of the internal bits of the analysis pictures corresponding to the normal graph group and the abnormal graph group are inconsistent, marking the data of the corresponding parts as non-qualified characteristic data; and sending the qualified feature data and the non-qualified feature data to the intelligent control unit together.
2. The image recognition-based robot control system of claim 1, wherein the monitoring process of the motion adjustment monitoring unit is as follows:
monitoring the operation of the robot hand, marking the robot hand with action adjustment as an adjustment analysis object, setting a mark k as a natural number more than 1, acquiring the frequency of the adjustment analysis object for action adjustment and the interval duration for action adjustment, and marking the frequency of the adjustment analysis object for action adjustment and the interval duration for action adjustment as PLk and SCk respectively; acquiring the frequency of the adjustment and analysis object for repeatedly performing the same action adjustment, and marking the frequency of the adjustment and analysis object for repeatedly performing the same action adjustment as TZk; analyzing and acquiring an action adjustment monitoring coefficient Xk of an adjustment analysis object, and comparing the action adjustment monitoring coefficient Xk of the adjustment analysis object with an action adjustment monitoring coefficient threshold value: if the action adjustment monitoring coefficient Xk of the adjustment analysis object exceeds the action adjustment monitoring coefficient threshold value, judging that the action adjustment monitoring of the corresponding adjustment analysis object is unqualified, generating an action adjustment abnormal signal and sending the action adjustment abnormal signal to an intelligent control unit, generating a pause operation instruction and sending the pause operation instruction to a server after the intelligent control unit receives the action adjustment abnormal signal, and performing shutdown maintenance on the corresponding manipulator by the server; and if the action adjustment monitoring coefficient Xk of the adjustment analysis object does not exceed the action adjustment monitoring coefficient threshold, judging that the action adjustment monitoring of the corresponding adjustment analysis object is qualified, generating an action adjustment normal signal and sending the action adjustment normal signal to the quality prediction unit.
3. The image recognition-based robot control system of claim 1, wherein the quality prediction process comprises:
collecting the product clamping qualification rate of the robot in the operation process and the repeated process frequency of the robot in the operation process, and respectively marking the product clamping qualification rate of the robot in the operation process and the repeated process frequency of the robot in the operation process as J and L; analyzing the quality prediction coefficient Y of the robot hand by analyzing and acquiring the quality prediction coefficient Y of the robot hand, and if the quality prediction coefficient Y of the robot hand is in an ascending trend, predicting the quality of the robot hand to be in a low-quality trend; and if the quality prediction coefficient Y of the robot is in a descending trend, predicting the quality of the robot to be in a high quality trend.
4. A robot control method based on image recognition, characterized by comprising a robot control system based on image recognition according to any one of claims 1 to 3.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210299746.1A CN114378835B (en) | 2022-03-25 | 2022-03-25 | Robot control system based on image recognition and control method thereof |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210299746.1A CN114378835B (en) | 2022-03-25 | 2022-03-25 | Robot control system based on image recognition and control method thereof |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114378835A CN114378835A (en) | 2022-04-22 |
| CN114378835B true CN114378835B (en) | 2022-06-17 |
Family
ID=81205067
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210299746.1A Active CN114378835B (en) | 2022-03-25 | 2022-03-25 | Robot control system based on image recognition and control method thereof |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114378835B (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107949865A (en) * | 2015-09-02 | 2018-04-20 | 富士通株式会社 | Abnormal detector, method for detecting abnormality and abnormality detecting program |
| CN111531580A (en) * | 2020-04-27 | 2020-08-14 | 武汉工程大学 | Vision-based multi-task robot fault detection method and system |
| CN111638407A (en) * | 2020-01-20 | 2020-09-08 | 武汉裕大华纺织有限公司 | Intelligent spinning cotton grabbing fault detection system |
| CN112549073A (en) * | 2020-11-18 | 2021-03-26 | 马鞍山因特莱信息科技有限公司 | Industrial robot performance detection system |
| WO2021064833A1 (en) * | 2019-09-30 | 2021-04-08 | 日本電気株式会社 | Abnormality detection device, abnormality detection method, and recording medium |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP3539728A1 (en) * | 2018-03-17 | 2019-09-18 | Tata Consultancy Services Limited | System and method for fault detection in robotic actuation |
-
2022
- 2022-03-25 CN CN202210299746.1A patent/CN114378835B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107949865A (en) * | 2015-09-02 | 2018-04-20 | 富士通株式会社 | Abnormal detector, method for detecting abnormality and abnormality detecting program |
| WO2021064833A1 (en) * | 2019-09-30 | 2021-04-08 | 日本電気株式会社 | Abnormality detection device, abnormality detection method, and recording medium |
| CN111638407A (en) * | 2020-01-20 | 2020-09-08 | 武汉裕大华纺织有限公司 | Intelligent spinning cotton grabbing fault detection system |
| CN111531580A (en) * | 2020-04-27 | 2020-08-14 | 武汉工程大学 | Vision-based multi-task robot fault detection method and system |
| CN112549073A (en) * | 2020-11-18 | 2021-03-26 | 马鞍山因特莱信息科技有限公司 | Industrial robot performance detection system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114378835A (en) | 2022-04-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN114819415B (en) | Power equipment fault prediction system based on data analysis | |
| CN111178587B (en) | Spark framework-based short-term power load rapid prediction method | |
| CN111461431A (en) | Optimization method and system based on screw locking process in mobile phone manufacturing | |
| CN116051340A (en) | Intelligent park equipment energy consumption detection system based on intelligent sensing technology | |
| CN111396248A (en) | Wind turbine generator set intelligent yaw control method based on short-term wind direction prediction | |
| CN115358626A (en) | Safety inspection early warning system for mineral processing production equipment | |
| CN116149282B (en) | Processing control method and system for steel smelting based on flow control | |
| CN115115090A (en) | Wind power short-term prediction method based on improved LSTM-CNN | |
| CN114742250A (en) | Numerical control equipment operation fault prediction system based on data analysis | |
| CN114936681A (en) | Carbon emission prediction method based on deep learning | |
| CN116781560A (en) | Industrial computer remote intelligent control system based on artificial intelligence | |
| CN114378835B (en) | Robot control system based on image recognition and control method thereof | |
| CN115497610A (en) | Medical equipment fault prediction system and method based on data analysis | |
| CN113934536B (en) | Data acquisition method facing edge calculation | |
| CN114995242A (en) | Remote control device for low-voltage circuit breaker of offshore power grid | |
| CN110750455A (en) | Intelligent online self-updating fault diagnosis method and system based on system log analysis | |
| CN114297265A (en) | Intelligent operation and maintenance method based on Internet of things technology | |
| CN207038185U (en) | Pressurized-water reactor nuclear power plant core power control device based on Takagi Sugeno fuzzy controls | |
| Kang et al. | Implementation of PV monitoring system using Python | |
| CN114862251A (en) | Casting production line management system with capacity management function | |
| CN102175934A (en) | Data collection method of wave recording module | |
| CN112257997B (en) | PCS conversion rate calculation method and system based on neural network | |
| CN107942658B (en) | Method and system for predicting rotation period of circular knitting machine by adopting adaptive filter | |
| CN113510711A (en) | Industrial robot action execution monitoring and regulating method based on artificial intelligence and cloud monitoring and regulating platform | |
| CN117134508B (en) | Multi-data fusion monitoring system of power distribution one-key centralized control device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |