CN113799118B - Coal mine search and rescue robot based on machine vision and motion control - Google Patents
Coal mine search and rescue robot based on machine vision and motion control Download PDFInfo
- Publication number
- CN113799118B CN113799118B CN202110353385.XA CN202110353385A CN113799118B CN 113799118 B CN113799118 B CN 113799118B CN 202110353385 A CN202110353385 A CN 202110353385A CN 113799118 B CN113799118 B CN 113799118B
- Authority
- CN
- China
- Prior art keywords
- data
- robot
- mine
- search
- rescue
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 239000003245 coal Substances 0.000 title claims abstract description 37
- 238000001514 detection method Methods 0.000 claims abstract description 64
- 230000009429 distress Effects 0.000 claims abstract description 22
- 238000004891 communication Methods 0.000 claims abstract description 16
- 238000012549 training Methods 0.000 claims abstract description 16
- 238000012545 processing Methods 0.000 claims abstract description 8
- 238000013461 design Methods 0.000 claims abstract description 6
- 230000007613 environmental effect Effects 0.000 claims abstract description 6
- 206010033799 Paralysis Diseases 0.000 claims abstract description 4
- 238000000034 method Methods 0.000 claims description 37
- 230000008569 process Effects 0.000 claims description 28
- 238000005259 measurement Methods 0.000 claims description 21
- 238000005070 sampling Methods 0.000 claims description 18
- 238000013528 artificial neural network Methods 0.000 claims description 16
- 238000004364 calculation method Methods 0.000 claims description 13
- 230000001960 triggered effect Effects 0.000 claims description 8
- 238000013145 classification model Methods 0.000 claims description 6
- 230000007246 mechanism Effects 0.000 claims description 4
- 238000009825 accumulation Methods 0.000 claims description 3
- 230000005540 biological transmission Effects 0.000 claims description 3
- 150000001875 compounds Chemical class 0.000 claims description 3
- 230000003313 weakening effect Effects 0.000 claims description 3
- 230000004083 survival effect Effects 0.000 abstract description 5
- 230000009286 beneficial effect Effects 0.000 abstract description 4
- 238000012360 testing method Methods 0.000 description 14
- 230000000630 rising effect Effects 0.000 description 12
- 230000001133 acceleration Effects 0.000 description 5
- 230000000694 effects Effects 0.000 description 5
- 238000005516 engineering process Methods 0.000 description 5
- 238000010586 diagram Methods 0.000 description 4
- 238000012937 correction Methods 0.000 description 3
- 238000007726 management method Methods 0.000 description 3
- 238000012544 monitoring process Methods 0.000 description 3
- 238000012935 Averaging Methods 0.000 description 2
- 238000006243 chemical reaction Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 238000002474 experimental method Methods 0.000 description 2
- 238000004519 manufacturing process Methods 0.000 description 2
- 230000005622 photoelectricity Effects 0.000 description 2
- 238000003491 array Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 231100000481 chemical toxicant Toxicity 0.000 description 1
- 238000005314 correlation function Methods 0.000 description 1
- 230000003247 decreasing effect Effects 0.000 description 1
- 238000005265 energy consumption Methods 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 238000005065 mining Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 239000003440 toxic substance Substances 0.000 description 1
Images
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
- B25J9/1694—Programme controls characterised by use of sensors other than normal servo-feedback from position, speed or acceleration sensors, perception control, multi-sensor controlled systems, sensor fusion
- B25J9/1697—Vision controlled systems
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J11/00—Manipulators not otherwise provided for
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J19/00—Accessories fitted to manipulators, e.g. for monitoring, for viewing; Safety devices combined with or specially adapted for use in connection with manipulators
- B25J19/02—Sensing devices
- B25J19/021—Optical sensing devices
- B25J19/023—Optical sensing devices including video camera means
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
- B25J9/1602—Programme controls characterised by the control system, structure, architecture
- B25J9/161—Hardware, e.g. neural networks, fuzzy logic, interfaces, processor
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
- B25J9/1656—Programme controls characterised by programming, planning systems for manipulators
- B25J9/1664—Programme controls characterised by programming, planning systems for manipulators characterised by motion, path, trajectory planning
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
- B25J9/1656—Programme controls characterised by programming, planning systems for manipulators
- B25J9/1664—Programme controls characterised by programming, planning systems for manipulators characterised by motion, path, trajectory planning
- B25J9/1666—Avoiding collision or forbidden zones
-
- E—FIXED CONSTRUCTIONS
- E21—EARTH DRILLING; MINING
- E21F—SAFETY DEVICES, TRANSPORT, FILLING-UP, RESCUE, VENTILATION, OR DRAINING IN OR OF MINES OR TUNNELS
- E21F11/00—Rescue devices or other safety devices, e.g. safety chambers or escape ways
Abstract
A coal mine search and rescue robot based on machine vision and motion control comprises the following steps that 1, environmental data of the coal mine search and rescue robot are collected; step 2, training a detection model of the people in distress in the mine; step 3, designing a motion control system of the coal mine search and rescue robot: step 4, designing a primary navigation system of the coal mine search and rescue robot; and 5, processing and storing data, wherein under the actual condition of mine disaster, a communication line is completely destroyed or partially paralyzed, and the communication cannot be used, so that the related data collected by the search and rescue robot are packaged in a certain grouping way, written into the SD card storage module according to the SPI communication protocol and used as return data support. The design of the invention enables underground data to be acquired by the robot deep field detection, which is beneficial to the planning and commanding of field rescue, ensures the survival rate of trapped people and reduces unnecessary loss and disasters.
Description
Technical Field
The invention relates to the field of robot control, in particular to a coal mine search and rescue robot based on machine vision and motion control.
Background
China has huge reserves of coal resources, and according to data, china is the largest consuming country and producing country in the coal field worldwide. Statistics show that in the total amount of energy industry in China, the amount of energy contributed by coal accounts for nearly 70% of the national primary energy consumption and production, and the proportion reaches more than 50% in 2050 according to backward estimation of the trend. Therefore, the exploitation and use of coal are a main energy support in China throughout the development in the future, and the coal industry developed from the main energy support needs related support and guarantee. One of the main aspects of the security is the rescue work of mine disaster. The coal production industry in China is developed, the yield is huge, the huge cardinality, the poor mine geology in China and the high-concentration gas in the mine inevitably lead China to be a high-occurrence place of mine disaster events, and the frequent mine disaster events cause huge troubles to the coal mining work.
After an underground mine disaster accident occurs, the underground condition is complex and changeable, multiple disasters occur, and dangerous environmental conditions and unknown field conditions are not suitable for being entered into the field by rescue workers for exploration and rescue at the first time. The coal mine in China has complex structure, high gas concentration and the like, and is not suitable for directly introducing related rescue robots from abroad as rescue auxiliary work. Nowadays, with the high development of modern electronic technology, various technologies are gradually improved, and the derived intelligent technology and robot technology enable a rescue scheme coordinated by intelligent equipment such as a robot. The rescue robot with the related functions can replace manpower to enter a dangerous environment to perform rescue work, so that the safety of personnel can be guaranteed, and various data conditions can be acquired professional more quickly.
The underground mine disaster condition is a complex and complex state, and the detected data comprises various harmful toxic chemical gas concentrations, site temperature, positions and states of trapped persons, whether the site has human living conditions or not, mine way deformation and collapse conditions and the like. The data are deeply detected and acquired on site through the robot, so that the planning and commanding of on-site rescue are facilitated, the survival rate of trapped people is guaranteed, and unnecessary loss and disasters are reduced.
Disclosure of Invention
In order to solve the problems, the invention provides a coal mine search and rescue robot based on machine vision and motion control, underground data is acquired through the robot deep field detection, people who are difficult to search and rescue are searched and rescued, the survival rate of trapped people is guaranteed, and unnecessary loss and disasters are reduced.
Step 1, collecting environmental data of a coal mine search and rescue robot: acquiring data of the path condition of the mine and the deformation condition of the mine along the way, and scanning the mine along the way by a steering engine matched with a distance sensor and a CCD (charge coupled device) sensor to obtain the deformation data of the mine;
step 2, training a detection model of the people in distress in the mine tunnel: respectively extracting the outline features of the image and the significance detection values of the outline features from the detected data set, and taking the significance detection fingers as input training BP neural network classification models;
step 3, designing a motion control system of the coal mine search and rescue robot: the vehicle body adopts a mode of driving four direct current motors, and reasonably adjusts the speed of the motors through a speed measuring device of a photoelectric encoder, PID control and PWM wave output, so that the accuracy of the advancing direction of the robot is guaranteed and the robot is prevented from deviating from a linear route;
step 4, designing a primary navigation system of the coal mine search and rescue robot: the system is a testability system, carries out navigation design on the robot, and is mainly divided into three parts, namely a distressed personnel detection system, an intelligent obstacle avoidance system and an automatic return system;
and 5, processing and storing the data, wherein under the actual condition of mine disaster, the communication line is completely destroyed or partially paralyzed, and the communication cannot be used, so that the relevant data collected by the search and rescue robot is packaged by certain grouping, written into the SD card storage module according to the SPI communication protocol and used as the data support of return voyage.
Further, in step 2, the process of training the mine disaster detection model can be expressed as follows:
step 2.1, acquiring data images in a mine road through a CCD sensor to form a training data image set;
step 2.2, extracting the contour features of the data image:
carrying out binarization processing on the acquired image data, and calculating the centroid (x) of the imagek,yk) And calculating the distance d from the image binary data to the centroidi:
Wherein (x)i,yi) i =1,2, …, n is binary image data;
step 2.3, extracting a significance detection value of the contour features:
ci＝cos(ri) (3)
Qi＝cos-1(si) (5)
in the formula (I), the compound is shown in the specification,
mean value, T, representing profile features1Is a significance detection threshold, riFeature data representing the mean value of the profile features, QiA saliency detection value representing a contour feature;
and 2.4, taking the contour feature significance detection value extracted from the training data set as input, taking the label of the victim target in the mine as output to train a three-layer BP neural network classification model, and obtaining the trained BP neural network victim detection model.
Further, the process of designing the motion control system of the coal mine search and rescue robot in step 3 can be represented as follows:
in order to realize the stable operation and real-time obstacle avoidance of the search and rescue robot, the invention uses a PID algorithm to control the motor of the robot, analyzes the output of the photoelectric speed measurement encoder, calculates the rotating speed value of the motor, and realizes the PID operation by taking the rotating speed value as feedback.
The main principle of PID control is to read control data from a controlled end, read the real-time rotating speed of a motor by a photoelectric coding disc, compare preset rotating speed parameters (PWM wave output duty ratio), make a difference e (t) between the two after calibration, add the proportional term, the integral term and the differential term of the deviation e (t) to obtain the duty ratio of the PWM wave, and send the duty ratio to an executing mechanism, and finally the adjusting equation of PID control is as follows:
wherein k isp，Ti，TDThe control parameters are proportional integral and differential of PID respectively, u (t) is the duty ratio output of the PWM wave of the robot at the time t, and the transmission function of PID control is as follows:
when the PID control is executed in the program, the PID is adjusted to be digital sampling, so that it needs to be discretized, and thus the control equation of the discrete PID control is:
in equation 8, u (n) is the real-time output state of the nth sampling time controller, e (n) is the deviation between the input quantity of the nth sampling time controller and the output quantity of the last time, and T is the sampling period.
The integral term of the discretization PID control mode needs to carry out accumulation calculation on the deviation e (i) of each moment before the nth moment, because some tiny inaccuracies exist in the measuring process, the tiny inaccuracies are continuously superposed in multiple times of measurement to cause large deviation, and meanwhile, continuous addition sampling calculation causes great burden calculation amount on an MCU, so that the method is not suitable for actual operation. In view of these disadvantages of the discrete PID control, the actual PID control operation mostly adopts incremental regulation, and the expression of the control mode is:
Du(n)＝u(n)-u(n-1) (9)
for the adjustment of PID parameters, the engineering setting method is relatively direct, a set of relatively suitable parameters can be directly obtained after one-time experiment, the operation difficulty is relatively low, and meanwhile, the parameters can be directly used without being tested. In the program, several main parameters are defined, then the sampling period is determined, and the proportional constant, the integral constant and the differential constant of PID control are preliminarily set for later testing the actual control condition and finally setting.
Then, the main routine sets a preset rotation speed, reads the actual rotation speed, enters a PID control routine to calculate (the calculation method refers to the formulas (10) and (11)), obtains a deviation correction amount, and corrects the duty ratio of the output PWM wave, thereby achieving the PID control target.
The PID control testing condition is monitored through repeated testing, and the specific operation method is to print real-time rotating speed information to the serial port of the upper computer, manually observe the information and determine the control effect. And repeatedly adjusting a proportional constant, an integral constant and a differential constant according to control problems reflected in the control effect, such as adjusting time, overshoot, stability and the like, thereby completing the engineering setting work. Particularly, in the test process, each parameter needs to be set under the condition of relatively actual load, so that the real-time adjustment and monitoring of the controlled object are realized.
Further, in step 4, the process of designing the primary navigation system of the coal mine search and rescue robot can be represented as follows:
a system of a motion control system comprising: the system comprises a victim detection system, an intelligent obstacle avoidance system and an automatic return system;
the detection system for the people in distress consists of a CCD sensor and a BP neural network people in distress detection model, the coal mine search and rescue robot obtains image data of a mine passage through the CCD sensor in the running process of the mine passage, meanwhile, after the step 2.2 and the step 2.3, the significance detection value of the image data is input into the BP neural network people in distress detection model to obtain the detection target of the people in distress in the mine passage, and if the detection model detects that the people in distress exist in the image data, emergency rescue work is started;
the intelligent obstacle avoidance system consists of ultrasonic ranging sensors and a motor, the ultrasonic ranging sensors at four different positions detect the distance of obstacles around the robot, reasonable judgment and decision are carried out on the conditions that the trolley travels to different obstacles according to preset numerical values, terminal point judgment is carried out on specific obstacles which cannot travel, and a return flight instruction is triggered;
the automatic return system mainly comprises an SD card, and during search and rescue of the robot, forward recording of traveling coordinate data is performed, and after a return instruction occurs, the automatic return system is activated; firstly, the information of coordinates along the way is reversely read from the SD card, real-time advancing coordinates are calculated through measurement and calculation of an MPU6050 and a speed measuring encoder, the robot is controlled to advance to the next target coordinate after the comparison with the coordinates, a path is corrected in real time, and the intelligent obstacle avoidance system conducts command in a weakening manner in the process, so that the advancing safety is guaranteed.
The coal mine search and rescue robot based on machine vision and motion control has the beneficial effects that: the invention has the technical effects that:
1. in the invention, the saliency detection is carried out on the features on the basis of the technology of extracting the outline features in the image, and compared with the method of directly inputting the features into a BP neural network, the model detection precision after the saliency detection is higher;
2. the invention solves the problem that dangerous environmental conditions and unknown field conditions are not suitable for the first time for rescuing people to enter the field to carry out exploration and rescue work.
3. The invention acquires the underground data by the robot deep field detection, is beneficial to the planning and commanding of field rescue, ensures the survival rate of trapped people, and reduces unnecessary loss and disasters.
Drawings
FIG. 1 is a hardware and logic block diagram of the motion control system of the present invention;
FIG. 2 is a hardware block diagram of the motion control system of the present invention;
FIG. 3 is a flow chart of the intelligent obstacle avoidance system of the present invention;
fig. 4 is a flow chart of the automatic return flight group.
Detailed Description
The invention is described in further detail below with reference to the following detailed description and accompanying drawings:
the invention provides a coal mine search and rescue robot based on machine vision and motion control, which aims to be beneficial to planning and commanding of on-site rescue, guarantee the survival rate of trapped people and reduce unnecessary loss and disasters.
Step 1, collecting environmental data of a coal mine search and rescue robot: acquiring path conditions of the mine and deformation condition data of the mine along the way, and scanning the mine along the way by a steering engine matched with a distance sensor and a CCD (charge coupled device) sensor to obtain the deformation data of the mine;
the mine data scanning system is the 'core work' of the robot, a plurality of settings of the motion control platform are all used for carrying the mine scanning system to enter a mine and executing the due tasks of the mine data scanning system, and the whole design task is used for obtaining the information of the mine after disaster and preparing planning work for rescue planning. Therefore, the mine scanning system is the central element of the whole system.
The mine tunnel scanning system mainly collects mine tunnel shape information of a path along the way, provides data support for drawing a 3D general diagram of the mine tunnel, also provides data support for rescue workers to judge the deformation condition of the mine tunnel, and simultaneously provides detection data for a detection model of people in distress. The main research steps are the drive of the steering engine, the CCD sensor and the infrared distance measurement scanning. The steering engine mechanism is used for driving a scanning device to perform tangent scanning on the mine, an infrared distance measuring module is used as the scanning device to detect distance (height) information of the steering engine in each rotating angle, and a CCD sensor collects images of the mine at regular time.
The steering engine drive is characterized in that the working mode of the steering engine is understood firstly, PWM waves with specific frequency and duty ratio are generated through a core chip, the test of the rotation fixed angle of the steering engine is completed, then the duty ratio is increased progressively and decreased progressively, and the steering engine is controlled to drive in a rotating mode.
The infrared ranging test comprises the steps of firstly testing ADC conversion, then matching reference voltages of a ranging module and an internal ADC conversion module, testing infrared ranging, then injecting a formula into a program to complete calculation from converted digital quantity to voltage value, and optimizing sampling (averaging, removing bad values and the like) logic to improve the accuracy of ranging data. And finally, joint adjustment is carried out on the steering engine and the infrared distance measurement module, and distance measurement sampling is activated at each angle to complete a set task.
Step 2, training a detection model of the people in distress in the mine tunnel: respectively extracting the outline characteristics of the image and the significance detection values of the outline characteristics from the detected data set, and taking significance detection fingers as input training BP neural network classification models;
step 2, the process of training the mine disaster detection model can be expressed as follows:
step 2.1, acquiring data images in a mine road through a CCD sensor to form a training data image set;
step 2.2, extracting the contour features of the data image:
carrying out binarization processing on the acquired image data, and calculating the centroid (x) of the imagek,yk) And calculating the distance d from the image binary data to the centroidi:
Wherein (x)i,yi) i =1,2, …, n is binary image data;
step 2.3, extracting a significance detection value of the contour features:
ci＝cos(ri) (3)
Qi＝cos-1(si) (5)
in the formula (I), the compound is shown in the specification,
mean value, T, representing a profile feature1Is a significance detection threshold, riRepresenting feature data after the mean value of the profile features, QiA saliency detection value representing a contour feature;
and 2.4, taking the contour feature significance detection value extracted from the training data set as input, taking the label of the victim target in the mine as output to train the three-layer BP neural network classification model, and obtaining the trained BP neural network victim detection model.
Step 3, designing a motion control system of the coal mine search and rescue robot: the vehicle body adopts a mode of driving four direct current motors, and reasonably adjusts the speed of the motors through a speed measuring device of a photoelectric encoder, PID control and PWM wave output, so that the accuracy of the advancing direction of the robot is guaranteed and the robot is prevented from deviating from a linear route;
the process of designing the motion control system of the coal mine search and rescue robot in the step 3 can be expressed as follows:
in order to realize the stable operation and real-time obstacle avoidance of the search and rescue robot, the invention uses a PID algorithm to control the motor of the robot, analyzes the output of the photoelectric speed measurement encoder, calculates the rotating speed value of the motor, and realizes the PID operation by taking the rotating speed value as feedback.
The main principle of PID control is to read control data from a controlled end, read the real-time rotating speed of a motor by a photoelectric coding disc, compare preset rotating speed parameters (PWM wave output duty ratio), make a difference e (t) between the two after calibration, add the proportional term, the integral term and the differential term of the deviation e (t) to obtain the duty ratio of the PWM wave, and send the duty ratio to an executing mechanism, and finally the adjusting equation of PID control is as follows:
wherein k isp，Ti，TDThe control parameters are proportional integral and differential of PID respectively, u (t) is the duty ratio output of the PWM wave of the robot at the time t, and the transmission function of PID control is as follows:
when the PID control is executed in the program, the PID is adjusted to be digital sampling, so it needs to be discretized, and thus the control equation of the discrete PID control is:
in formula 8, u (n) is the real-time output state of the nth sampling time controller, e (n) is the deviation between the input quantity of the nth sampling time controller and the output quantity of the last time, and T is the sampling period.
The integral term of the discretization PID control mode needs to accumulate and calculate the deviation e (i) of each moment before the nth moment, and since some tiny inaccuracies exist in the measuring process, the tiny inaccuracies are continuously superposed in multiple measurements to cause large deviation, and meanwhile, continuous adding sampling calculation causes great burden calculation amount to an MCU, so that the method is not suitable for actual operation. In view of these disadvantages of the discrete PID control, in the actual PID control operation, incremental regulation is often used, and the expression of the control mode is:
Du(n)＝u(n)-u(n-1) (9)
for the adjustment of PID parameters, the engineering setting method is relatively direct, a set of relatively suitable parameters can be directly obtained after one-time experiment, the operation difficulty is relatively low, and meanwhile, the parameters can be directly used without being tested. In the program, the invention firstly defines several main parameters, then determines the sampling period, and carries out initial setting on the proportional constant, the integral constant and the differential constant of PID control, so as to test the actual control condition later and finally carry out setting.
Then, the main routine sets a preset rotation speed, reads the actual rotation speed, enters a PID control routine to calculate (the calculation method refers to the formulas (10) and (11)), obtains a deviation correction amount, and corrects the duty ratio of the output PWM wave, thereby achieving the PID control target.
The PID control testing condition is monitored through repeated testing, and the specific operation method is to print real-time rotating speed information to the serial port of the upper computer, manually observe the information and determine the control effect. And repeatedly adjusting a proportional constant, an integral constant and a differential constant according to control problems reflected in the control effect, such as adjusting time, overshoot, stability and the like, thereby completing the engineering setting work. In particular, in the above test process, each parameter needs to be set under a relatively actual load condition, so as to realize real-time adjustment and monitoring of the controlled object.
Step 4, designing a primary navigation system of the coal mine search and rescue robot: the system is a testability system, carries out navigation design on the robot, and is mainly divided into three parts, namely a distressed personnel detection system, an intelligent obstacle avoidance system and an automatic return system;
in step 4, the process of designing the primary navigation system of the coal mine search and rescue robot can be represented as follows:
the system of the motion control system of the present invention comprises: the system comprises a distress person detection system, an intelligent obstacle avoidance system and an automatic return system. The motion control system consists of four direct current motors, two motor driving modules, four ultrasonic ranging modules, an MPU6050 three-axis sensor and a CCD sensor, and the hardware and logic of the motion control system are formed as shown in figure 1. And selecting STM32F103ZET6 as a core control chip of the task in consideration of the control task. In order to read the real-time speed of the trolley and finally perform PID control, the output of the photoelectric speed measurement coding disc needs to be analyzed, the period of the output pulse is the period of the photoelectric speed measurement coding disc receiving the optical signal, namely the rotation period of the motor, so that the rising edge of the pulse is continuously triggered, captured and calculated, and the rotating speed value can be finally calculated.
The advanced timer and the basic timer of the STM32F103ZET6 have an input capture function, specifically, the level of the port is detected through the input port of the timer, the count value (TIMx _ CNT) in the real-time timer is stored into the corresponding input/comparison register (TIMx _ CCRx) at the moment when the level changes, namely, the moment of transmitting the level jump in real time can be recorded, and the hardware configuration diagram of the motion control system of the coal mine search and rescue robot is shown in figure 2.
In the speed measurement of the photoelectric coded disk, the real-time rotating speed of a certain motor can be obtained in real time by detecting the speed of output pulses and combining a calculation formula, so that data support is provided for PID control. The method comprises the steps of monitoring the occurrence time of the same level jump mode on an input port, calculating the time difference of two adjacent recording times according to the clock frequency of a timer after continuously recording a plurality of times, filtering bad values of a plurality of time differences, averaging to obtain a pulse period, and completing speed measurement.
The motor drive module samples the L298N direct current motor driver, and the driver is connected with the driver through an external 12V power supply source through STM32F103ZET6 four-way output PWM waves (two ways are one group), so that the PWM wave energy is enhanced, the processed PWM waves are four-way output (two ways are one group), and the processed PWM waves are connected with a follow-up motor to drive the motor to work. The used direct current motor of this patent is installed around fixed chassis, including wheel and photoelectricity encoding disc that tests the speed for the mobile device removes, and the main device is four direct current motor and photoelectricity encoding disc that tests the speed. Four groups (each group of two paths of PWM waves) are connected to the trolley platform to drive four motors to rotate and drive the platform to move. The motor speed is controlled by the duty ratio, one of the two PWM waves is set to be zero to control the motor to rotate forward and backward, for example, one PWM wave with the duty ratio of 4/9 is output, and the other PWM wave with the duty ratio of 0/9 is output.
For the output pulse signals carrying real-time rotating speed, when a first rising edge signal comes, a timer counting register (TIMx _ CNT) is set to be zero, then when each rising edge comes, the value in an input/comparison register (TIMx _ CCRx) is read and recorded in an array by using a correlation function, after five rising edges are continuously read (including the first rising edge), the value in the array is subjected to difference to obtain a time difference, and after an excessive bad value of deviation amount is removed, the remaining value is averaged to be used as a rotating speed information value.
And connecting the control chip, the driver and the motor according to rules of a timer output pin, a timer input pin and the like. The pins of the chip assigned to the motor drive groups are shown in table 1, taken into account as a whole.
TABLE 1 Motor drive group Pin assignment
And (4) distributing pins according to the table 1, and connecting a motor driving module and a power driving module.
The detection system for the people in distress consists of a CCD sensor and a BP neural network people in distress detection model, the coal mine search and rescue robot obtains image data of a mine passage through the CCD sensor in the running process of the mine passage, meanwhile, after the step 2.2 and the step 2.3, the significance detection value of the image data is input into the BP neural network people in distress detection model to obtain the detection target of the people in distress in the mine passage, and if the detection model detects that the people in distress exist in the image data, emergency rescue work is started.
The intelligent obstacle avoidance system is composed of ultrasonic ranging sensors and a motor, the ultrasonic ranging sensors at four different positions detect the distance of obstacles around the robot, the condition that the trolley travels to different obstacles is reasonably judged and decided according to preset numerical values, the terminal point judgment is made on the specific obstacle which cannot travel, and a return flight instruction is triggered. The intelligent obstacle avoidance system is as follows:
(1) Timer input capture
Different from the input capture logic of the photoelectric speed measurement coded disc in the motor drive group, the photoelectric speed measurement coded disc needs to continuously detect the interval time of the same trigger condition (rising edge trigger/falling edge trigger) according to the period of pulse wave when reading speed information, but needs to measure the duration time of high level in a pulse in the ultrasonic ranging detection of automatic obstacle avoidance.
The distance data obtained by multiplying the time length of the high level of the pulse wave measured at the input end of the ultrasonic pulse wave by the sound velocity in the real-time environment (the main determinant of the sound velocity is the real-time environment temperature which can be measured by the MPU 6050) is the obstacle distance in the direction acquired by the ultrasonic ranging module at the moment.
In order to complete the measurement of the high duration within a pulse, it is necessary to change the capture condition of the input capture, which needs to be set to a rising edge trigger first due to the characteristics of the output signal of the ultrasonic ranging module, reset the value in the timer count register (TIMx _ CNT) when a rising edge is detected, and set the trigger condition to a falling edge trigger. When the falling edge is detected, the normal operation input capture program stores the value in the counter register (TIMx _ CNT) into the corresponding input/comparison register (TIMx _ CCRx), reads the value in the input/comparison register (TIMx _ CCRx), calculates the time length according to the working frequency of the corresponding timer, and finally calculates the distance information.
In the programming, for a single group of ultrasonic ranging modules, only an input capture mode needs to be set, the ultrasonic modules are started, and whether the ultrasonic ranging modules are triggered by a rising edge or a falling edge is judged in the interrupt service function when the corresponding channel rises. After a determination of a rising edge, the value in the timer count register (TIMx _ CNT) is reset and the trigger condition is set to falling edge triggered using the independent polarity set function, clearing the corresponding interrupt flag bit. When the falling edge of the corresponding channel arrives, judging whether the function is triggered by the rising edge or the falling edge in the interrupt service function, after the function is determined to be the falling edge, reading the function by using an input/comparison register (TIMx _ CCRx), reading the value in the register, calculating the time difference value, storing the time difference value into a preset distance information register, setting the trigger condition as rising edge trigger, clearing the corresponding interrupt flag bit, and starting the next detection.
For the ultrasonic ranging modules with multiple groups, data classification needs to be performed by using arrays, single-group ranging work is performed in sequence, and a cross-group triggering interrupt service function is prevented, and a specific operation process and a flow chart thereof are shown in fig. 3.
(2) Automatic obstacle avoidance logic
The automatic obstacle avoidance logic is mainly based on distance information read by the ultrasonic ranging module, compares preset distance values, and issues an obstacle avoidance command (in a PWM wave form) to the motor driving group according to a preset scheme. The obstacle information of the loading position of the ultrasonic ranging module and the obstacle avoidance scheme thereof are shown in table 2. (namely, the first ultrasonic ranging module, the second ultrasonic ranging module, the third ultrasonic ranging module, and the fourth ultrasonic ranging module)
TABLE 2 obstacle avoidance scheme
For the obstacle right before, in this state, the forced 15-degree left (right) steering is performed first (the MPU6050 monitors the steering angle) to prevent the continuous buffeting steering, the current situation is detected, and if the obstacle right before is still in the obstacle state, the forced 15-degree left (right) steering is continued, and the above processes are circulated to search for the angle which can be advanced. And if the forced left (right) steering is still in the front obstacle state, continuously forcing 15-degree right (left) steering, and repeating the above processes to search for angles capable of advancing. The terminal point determination is made when the forced left (right) steering accumulation is still unable to proceed for 90 degrees (the "straight ahead" obstacle condition).
For the left side and the right side, a threshold area is added for judging the steering caused by the difference value, namely the difference value between the left side and the right side is larger than 10cm to trigger a steering command, so that continuous jitter steering is prevented. And when any value of the two is less than 10cm, judging the end point. The end point judgment means that the robot judges that the robot reaches a position where the robot cannot move forward, and an automatic return program is triggered.
The automatic return system mainly comprises an SD card, and in the process of searching and rescuing by the robot, forward recording of traveling coordinate data is performed, and after a return instruction occurs, the automatic return system is activated. Firstly, the information of coordinates along the way is reversely read from an SD card, real-time advancing coordinates are calculated through measurement and calculation of an MPU6050 and a speed measuring encoder, the robot is controlled to advance to the next target coordinate after the comparison with the coordinates, a path is corrected in real time, and the intelligent obstacle avoidance system conducts command in a weakening mode (the recognition standard of the warning distance is reduced) in the process, so that the advancing safety is guaranteed.
The automatic return flight needs to position the robot in real time, and when the robot is positioned in real time, besides the distance information calculated by reading speed information by using a photoelectric speed measurement coding disc, the robot also needs to perform attitude detection to calculate the advancing angle. The MPU6050 can execute the angle detection task, and the chip is connected with the main control chip in an IIC communication mode.
(1) MPU6050 six-axis sensor
MPU6050 is a global capital-integrated 6-axis motion processing component, which is introduced by InvenSense corporation. A3-axis gyroscope, a 3-axis acceleration sensor and a second IIC interface integrated on a hardware board are arranged in a chip of the sensor, and the sensor can be used for connecting an external magnetic sensor.
The MPU6050 integrates and calculates angle information by using a hardware acceleration engine with a Digital Motion Processor (DMP), and then outputs complete 9-axis attitude fusion calculation data to an application end through a main IIC communication interface.
The design does not integrate an external magnetic sensor, and only adopts an internal 6-axis sensor to solve the motion attitude, so that only a main IIC interface is used in data communication. And then, the AD0 is used for carrying out address setting, the chip selection pin is selected for a long time by default, and an interrupt signal generated by the MPU6050 is not used.
In the MPU6050 initialization, the IIC communication interface is initialized, the reset operation is performed on the MPU6050 (controlled by the power management register 1 (0X 6B)), then the full-range measurement range of the angular velocity sensor and the acceleration sensor is set (set by the gyro configuration register (0X 1B) and the acceleration sensor configuration register (0X 1C)), then some other registers are set as required (for example, the interrupt enable register (0X 38), the user control register (0X 6A), the FIFO enable register (0X 23), the sampling rate division register (0X 19), the configuration register (0X 1A), and the like), then the system clock (the power management register 1 (0X 6B)) is set, the PLL of the X-axis gyro is generally used as a clock source, and after the basic setting is completed, the angular velocity sensor (gyro) and the acceleration sensor (configured by the power management register 2 (0X 6C)) are finally started.
(2) Automatic return logic
The programming of the automatic return journey needs to acquire coordinate information of the robot before the return journey in the advancing process, so that the real-time speed information (provided by a photoelectric speed measurement coding disc) and angle information (provided by an MPU 6050) need to be read in the robot advancing process.
The recording mode of the path is carried out in a mode of direct curve instead of curved, only two-dimensional traveling information, namely direction angle information, is recorded, steering information and a navigation coordinate position (namely a real-time coordinate at the moment) are recorded at the position where each direction angle changes, and the real-time coordinate position is updated instantly through real-time speed information and angle information.
And between two navigation coordinate recording points, timing by using a basic timer, calculating the distance by multiplying the real-time speed, accumulating the small distances to be recorded as the distance between two coordinate points, and sequentially writing the distance and the navigation coordinate into the SD card.
Particularly, the sector position information of the SD card needs to be updated in real time when the navigation information group is written, so that the reverse order information reading can be correctly performed during the return voyage.
When the terminal point is determined (provided by the obstacle avoidance group), a return navigation program is started, a navigation information group is read from the SD card, the robot turns by 180 degrees and starts to navigate according to coordinates and coordinate distance information, and it is noted that angle information read from the SD card needs to be calculated twice.
Meanwhile, the instant coordinate position is still in a real-time updating state, whether the position information is correct or not is verified at each steering coordinate, the coordinates of the two are calculated, and if the straight line deviation distance exceeds 30cm, correction operation is carried out. The above process is circulated, the coordinate (0.0) position, namely the robot drop point, is returned, and the automatic return flight group flowchart is shown in fig. 4.
And 5, processing and storing the data, wherein under the actual condition of mine disaster, the communication line is completely destroyed or partially paralyzed, and the communication cannot be used, so that the relevant data collected by the search and rescue robot is packaged by certain grouping, written into the SD card storage module according to the SPI communication protocol and used as the data support of return voyage.
The above description is only a preferred embodiment of the present invention, and is not intended to limit the present invention in any way, but any modifications or equivalent variations made according to the technical spirit of the present invention are within the scope of the present invention as claimed.
Claims (1)
1. The utility model provides a colliery search and rescue robot based on machine vision and motion control which concrete step as follows, its characterized in that:
step 1, collecting environmental data of a coal mine search and rescue robot: acquiring data of the path condition of the mine and the deformation condition of the mine along the way, and scanning the mine along the way by a steering engine matched with a distance sensor and a CCD (charge coupled device) sensor to obtain the deformation data of the mine;
step 2, training a detection model of the people in distress in the mine tunnel: respectively extracting the outline features of the image and the significance detection values of the outline features from the detected data set, and taking the significance detection fingers as input training BP neural network classification models;
step 2, the process of training the detection model of the people in distress in the mine path is represented as follows:
step 2.1, acquiring data images in a mine road through a CCD sensor to form a training data image set;
step 2.2, extracting the contour features of the data image:
carrying out binarization processing on the acquired image data, and calculating the centroid (x) of the imagek,yk) And calculating the distance d from the image binary data to the centroidi:
Wherein (x)i,yi)i＝1,2, …, n is binary image data;
step 2.3, extracting a significance detection value of the contour features:
ci＝cos(ri) (3)
Qi＝cos-1(si) (5)
in the formula (I), the compound is shown in the specification,
mean value, T, representing a profile feature1Is a significance detection threshold, riRepresenting feature data after the mean value of the profile features, QiA saliency detection value representing a contour feature;
step 2.4, taking the contour feature significance detection value extracted from the training data set as input, taking the label of the victim target in the mine as output to train a three-layer BP neural network classification model, and obtaining a trained BP neural network victim detection model;
step 3, designing a motion control system of the coal mine search and rescue robot: the vehicle body adopts a mode of driving four direct current motors, and reasonably adjusts the speed of the motors through a speed measuring device of a photoelectric encoder, PID control and PWM wave output, so that the accuracy of the advancing direction of the robot is guaranteed and the robot is prevented from deviating from a linear route;
the process of designing the motion control system of the coal mine search and rescue robot in the step 3 is represented as follows:
in order to realize the stable operation and real-time obstacle avoidance of the search and rescue robot, a PID algorithm is used for controlling a motor of the robot, the output of a photoelectric speed measurement encoder is used for analyzing, the rotating speed value of the motor is calculated, and the rotating speed value is used as feedback to realize the PID operation;
PID control is to read control data from a controlled end, read the real-time rotating speed of a motor by a photoelectric coding disc, compare preset PWM wave duty ratio rotating speed parameters, make difference e (t) after calibration, add proportional term, integral term and differential term of the deviation e (t) to obtain the duty ratio value of the PWM wave, and send the duty ratio value to an actuating mechanism, and finally the adjusting equation of the PID control is as follows:
wherein k isp，Ti，TDThe control parameters are proportional integral and differential of PID respectively, u (t) is the duty ratio output of the PWM wave of the robot at the time t, and the transmission function of PID control is as follows:
when the PID control is executed in a program, PID is adjusted into digital samples, and discretization operation is carried out on the digital samples, so that the control equation of the discrete PID control is as follows:
in formula 8, u (n) is the real-time output state of the nth sampling time controller, e (n) is the deviation between the input quantity of the nth sampling time controller and the output quantity of the last time controller, and in the formula, T is the sampling period;
the integral term of the discretization PID control mode needs to carry out accumulation calculation on the deviation e (i) of each moment before the nth moment, and incremental adjustment is adopted because of the existence of accumulated deviation in the measurement process, and the expression of the control mode is as follows:
Du(n)＝u(n)-u(n-1) (9)
for the PID parameter adjustment, a control parameter is obtained by adopting an engineering setting method;
step 4, designing a primary navigation system of the coal mine search and rescue robot: the system is a testability system, carries out navigation design on the robot, and is divided into three parts, namely a distressed person detection system, an intelligent obstacle avoidance system and an automatic return system;
in step 4, the process of designing the primary navigation system of the coal mine search and rescue robot is represented as follows:
a system of a motion control system comprising: the system comprises a victim detection system, an intelligent obstacle avoidance system and an automatic return system;
the detection system for the people in distress consists of a CCD sensor and a BP neural network people in distress detection model, the coal mine search and rescue robot obtains image data of a mine passage through the CCD sensor in the running process of the mine passage, meanwhile, after the step 2.2 and the step 2.3, the significance detection value of the image data is input into the BP neural network people in distress detection model to obtain the detection target of the people in distress in the mine passage, and if the detection model detects that the people in distress exist in the image data, emergency rescue work is started;
the intelligent obstacle avoidance system consists of ultrasonic ranging sensors and a motor, the ultrasonic ranging sensors at four different positions detect the distance of obstacles around the robot, reasonable judgment and decision are carried out on the conditions that the trolley travels to different obstacles according to preset numerical values, terminal point judgment is carried out on specific obstacles which cannot travel, and a return flight instruction is triggered;
the automatic return system mainly comprises an SD card, and during the search and rescue process of the robot, firstly, forward recording of traveling coordinate data is carried out, and after a return instruction occurs, the automatic return system is activated; firstly, reversely reading along-way coordinate information from an SD card, measuring and calculating through an MPU6050 and a speed measuring encoder, calculating a real-time advancing coordinate, controlling the robot to advance to a next target coordinate after comparing the real-time advancing coordinate with the coordinate, correcting a path in real time, and conducting command by weakening an intelligent obstacle avoidance system in the process to ensure the advancing safety;
and 5, processing and storing the data, wherein under the actual condition of mine disaster, the communication line is completely destroyed or partially paralyzed, and the communication cannot be used, so that the relevant data collected by the search and rescue robot is packaged by certain grouping, written into the SD card storage module according to the SPI communication protocol and used as the data support of return voyage.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110353385.XA CN113799118B (en) | 2021-04-01 | 2021-04-01 | Coal mine search and rescue robot based on machine vision and motion control |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110353385.XA CN113799118B (en) | 2021-04-01 | 2021-04-01 | Coal mine search and rescue robot based on machine vision and motion control |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113799118A CN113799118A (en) | 2021-12-17 |
| CN113799118B true CN113799118B (en) | 2022-11-01 |
Family
ID=78892921
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110353385.XA Active CN113799118B (en) | 2021-04-01 | 2021-04-01 | Coal mine search and rescue robot based on machine vision and motion control |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113799118B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116189229B (en) * | 2022-11-30 | 2024-04-05 | 中信重工开诚智能装备有限公司 | Personnel tracking method based on coal mine auxiliary transportation robot |
| CN116682250B (en) * | 2023-06-06 | 2024-02-13 | 深圳启示智能科技有限公司 | Robot wireless remote control device |
Family Cites Families (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CA2407992C (en) * | 2000-05-01 | 2010-07-20 | Irobot Corporation | Method and system for remote control of mobile robot |
| CN102053249B (en) * | 2009-10-30 | 2013-04-03 | 吴立新 | Underground space high-precision positioning method based on laser scanning and sequence encoded graphics |
| CN103148812B (en) * | 2013-02-06 | 2016-03-02 | 中联重科股份有限公司 | Tunnel contour scanning device, method and comprise the engineering machinery of this equipment |
| CN203702258U (en) * | 2014-02-14 | 2014-07-09 | 西安煤航卫星数据应用有限公司 | Data collection system for mine tunnel three-dimensional modeling |
| CN105216905B (en) * | 2015-10-27 | 2018-01-02 | 北京林业大学 | Immediately positioning and map building exploration search and rescue robot |
| CN105547288A (en) * | 2015-12-08 | 2016-05-04 | 华中科技大学 | Self-localization method and system for mobile device in underground coal mine |
| CN107655898B (en) * | 2017-10-10 | 2023-11-03 | 山西省智慧交通研究院有限公司 | Stereoscopic scanning robot for road tunnel detection and implementation method thereof |
| RU2682298C1 (en) * | 2018-01-22 | 2019-03-18 | Дмитрий Николаевич Саломатов | Robotized, mobile, modular mine-rescue complex and methods of application thereof |
| CN109002783A (en) * | 2018-07-02 | 2018-12-14 | 北京工业大学 | Rescue the human testing in environment and gesture recognition method |
| CN211076291U (en) * | 2019-09-25 | 2020-07-24 | 深圳市蔚蓝方舟科技有限公司 | Water rescue robot with original return function |
| CN112109090A (en) * | 2020-09-21 | 2020-12-22 | 金陵科技学院 | Multi-sensor fusion search and rescue robot system |
| CN112207821B (en) * | 2020-09-21 | 2021-10-01 | 大连遨游智能科技有限公司 | Target searching method of visual robot and robot |
-
2021
- 2021-04-01 CN CN202110353385.XA patent/CN113799118B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN113799118A (en) | 2021-12-17 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Brossard et al. | RINS-W: Robust inertial navigation system on wheels | |
| CN108536149B (en) | Unmanned vehicle obstacle avoidance control method based on Dubins path | |
| CN107246876B (en) | Method and system for autonomous positioning and map construction of unmanned automobile | |
| CN113799118B (en) | Coal mine search and rescue robot based on machine vision and motion control | |
| KR101997436B1 (en) | Object position measurement by car camera using vehicle motion data | |
| US8873832B2 (en) | Slip detection apparatus and method for a mobile robot | |
| EP2209091B1 (en) | System and method for object motion detection based on multiple 3D warping and vehicle equipped with such system | |
| CN105953796A (en) | Stable motion tracking method and stable motion tracking device based on integration of simple camera and IMU (inertial measurement unit) of smart cellphone | |
| CN105716617B (en) | The system and method for driving locus is drawn based on vehicle data | |
| CN105928531A (en) | Method for generating route accurately used for pilotless automobile | |
| CN110615017A (en) | Rail transit automatic detection system and method | |
| CN105865461A (en) | Automobile positioning system and method based on multi-sensor fusion algorithm | |
| CN110726409A (en) | Map fusion method based on laser SLAM and visual SLAM | |
| CN111947644B (en) | Outdoor mobile robot positioning method and system and electronic equipment thereof | |
| CN113593284B (en) | Method and device for planning path of vehicle in mine roadway and electronic equipment | |
| CN109325390B (en) | Positioning method and system based on combination of map and multi-sensor detection | |
| JP2019112049A (en) | Method for recognizing driving style of driver of land vehicle and corresponding device | |
| CN105387858B (en) | A kind of yacht intelligent guidance system and working method | |
| CN109540143B (en) | Pedestrian unconventional action direction identification method based on multi-sensing-source dynamic peak fusion | |
| CN103217154A (en) | Method and device for locating underground personnel in coal mine | |
| CN111077890A (en) | Implementation method of agricultural robot based on GPS positioning and automatic obstacle avoidance | |
| CN107600073A (en) | A kind of vehicle centroid side drift angle estimating system and method based on Multi-source Information Fusion | |
| CN114527752A (en) | Accurate positioning method for detection data of track inspection robot in low satellite signal environment | |
| CN113465728A (en) | Terrain perception method, terrain perception system, storage medium and computer equipment | |
| CN113029148A (en) | Inertial navigation indoor positioning method based on course angle accurate correction |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20231102 Address after: No. 6, Group 5, Xuehe Village, Qintong Town, Jiangyan District, Taizhou City, Jiangsu Province, 225500 Patentee after: Taizhou Huayiyuan Machinery Co.,Ltd. Address before: No. 99 Jiangning Road, Nanjing District hirokage 210000 cities in Jiangsu Province Patentee before: JINLING INSTITUTE OF TECHNOLOGY |
|
| TR01 | Transfer of patent right |