CN110977960A - Visual following system of three drive arms - Google Patents

Visual following system of three drive arms Download PDF

Info

Publication number
CN110977960A
CN110977960A CN201911057970.4A CN201911057970A CN110977960A CN 110977960 A CN110977960 A CN 110977960A CN 201911057970 A CN201911057970 A CN 201911057970A CN 110977960 A CN110977960 A CN 110977960A
Authority
CN
China
Prior art keywords
mechanical arm
target
motor
lower computer
value
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201911057970.4A
Other languages
Chinese (zh)
Inventor
张雷雨
王洪刚
杨斯钦
雷杨浩
张得阳
彭程
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing University of Technology
Original Assignee
Beijing University of Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing University of Technology filed Critical Beijing University of Technology
Priority to CN201911057970.4A priority Critical patent/CN110977960A/en
Publication of CN110977960A publication Critical patent/CN110977960A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J9/00—Programme-controlled manipulators
    • B25J9/16—Programme controls
    • B25J9/1694—Programme controls characterised by use of sensors other than normal servo-feedback from position, speed or acceleration sensors, perception control, multi-sensor controlled systems, sensor fusion
    • B25J9/1697—Vision controlled systems
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J9/00—Programme-controlled manipulators
    • B25J9/16—Programme controls
    • B25J9/1602—Programme controls characterised by the control system, structure, architecture
    • B25J9/161—Hardware, e.g. neural networks, fuzzy logic, interfaces, processor
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]

Abstract

The invention provides a vision following system of a three-drive mechanical arm, which comprises three parts, namely an upper computer vision identification part, an upper computer and lower computer communication part and a lower computer drive control part. The provided visual following system of the three-drive mechanical arm is characterized in that the upper computer visually identifies a frequency domain filtering mode based on convolution, predicts the position of a target possibly appearing in the next frame, mainly extracts the change coordinate of the target position by using the minimum output mean square error and the MOSSE algorithm, and sends the change coordinate to the lower computer; the upper computer and the lower computer are communicated by zigbee; the lower computer receives the command of the upper computer through the zigbee and sends a specified hexadecimal code to the driver to drive the corresponding motor accordingly, and the following action of the mechanical arm is completed through combination.

Description

Visual following system of three drive arms
Technical Field
The invention relates to a visual following system, in particular to a visual following system of a mechanical arm.
Background
Machine vision is an integral and important component of artificial intelligence, and develops with the development of artificial intelligence. Machine vision refers to the use of optical non-contact sensing devices to receive images of real scenes to obtain information to control a machine or process. The machine vision is frequently integrated with an automatic control device in a system to jointly complete a relatively complex control task with object identification and judgment, and the mechanical arm following system based on the machine vision plays a very important decisive role in the fields of operation of an industrial robot, grabbing of a mechanical arm and the like, and particularly needs to have high enough precision to complete corresponding tasks in the production process with high precision, high quality and multiple batches; therefore, the mechanical arm following system based on machine vision has great influence on the working efficiency of the machine, and the system can greatly reduce the workload of manual control and greatly improve the working precision for the production of special equipment.
At present, in the use process of the existing mechanical arm driving system based on machine vision, the following effect is poor, the traditional following system is only simple in image recognition and follows, the precision is poor, the limitation is large, and the mechanical arm driving system cannot be suitable for different place environments. Because the traditional three-dimensional positioning technology is used for establishing a three-dimensional coordinate system of a target by acquiring accurate three-dimensional coordinates of the target, in actual life, an object shakes or moves at a high speed due to external force factors, and a certain error or even an identification failure can be caused by the existing machine vision following system. Therefore, we propose a vision following system of three-drive mechanical arm to solve the above-mentioned problems.
Disclosure of Invention
The invention mainly solves the problem that the existing three-drive mechanical arm vision tracking system has defects, and provides an efficient, accurate and universal mechanical arm vision tracking system for a three-motor-driven mechanical arm. The upper computer vision recognition system of the system adopts the idea of relevant filtering (the more relevant two target relevant values are larger, namely the video frame is provided withThe more similar the initialized target is, the larger the obtained response is), the convolution theorem (the time domain convolution is equivalent to frequency domain multiplication, and the frequency domain convolution is equivalent to time domain multiplication) is applied to process the image collected by the camera, the target function of the MOSSE filter is searched, the closed solution is solved, and the peak value coordinate is updated to realize target tracking, so that a very efficient visual identification program is formed, and the updated position coordinate X is respectively extracted1And Y1Comparing the target position with the central coordinates (300, 100), obtaining a result, outputting the result to a lower computer, obtaining a motion track of the mechanical arm, and driving three motors to enable the mechanical arm to reach the target position; for the motor drive of the lower computer, a register is configured in the microprocessor to compile a read-write driver function, the communication between the driver and the microprocessor is completed through an RS485 serial port, and the current position information of the mechanical arm is fed back to the microprocessor in real time through an encoder. The read/write driver function achieves control by changing the state of the flag bit in the register. The specific implementation is that after receiving the instruction of the upper computer, the MODBUS protocol is written into the register by writing the function of the driver according to the bit, and is sent to the driver through the serial port, and the host computer sets the parameter of the driver and controls the operation through the read-write register function of the MODBUS.
Therefore, the invention adopts the following technical scheme and implementation steps:
determining a target is finding a filter h that maximizes its response at the target. f denotes the training image, g denotes the output image, h denotes the filter, F, G, H corresponds to its frequency domain value. And deducing a calculated value of the target H through a time domain formula and a frequency domain formula, and completing the conversion of the time domain convolution and the frequency domain convolution. And solving a closed solution of H by using the objective function of the filter and updating the MOSSE. Calling a python corresponding library function to generate an ideal response of a Gaussian shape, solving a target frame area and performing filter convolution to obtain a response value, extracting a corresponding maximum value as a new target position, updating a position coordinate new target as a center selection target, finally extracting updated X and Y values respectively, comparing and sending the updated X and Y values to a lower computer. The execution device of the lower computer is a servo motor, the servo motor selects a speed mode, the upper computer sends a driving message of the motor to the microprocessor through a wireless transmission serial port, and the microprocessor reads data based on a 485 communication protocol and writes the driving message into a motor driver to realize the adjustment of the rotating speed of the servo motor. The three motors respectively control different polar coordinate parameters to realize space motion. The upper computer converts the comparison result of the object (the deviation degree of the target relative to the center of the visual field) into the specific displacement direction and movement time (given speed) of the lower computer, so that the mechanical arm can adjust the position and the posture along with the movement of the target.
After a camera acquires a video image, selecting a target to be tracked in the video, then performing Gaussian filtering on a first frame acquired after selection, mapping the image and a filter onto a topological structure, filling a boundary in a circulating image mode, namely connecting the left edge of the image to the right edge, connecting the top to the bottom, and finally performing point-by-point cosine window processing to enable the edge of the image to become zero.
In order to convert the spatial domain correlation filtering to the frequency domain filtering to improve the operation speed, a convolution operator is introduced, wherein when the conversion of the point product of the time domain convolution and the frequency domain is carried out:
g＝f*h (1)
G＝F·H*(2)
Figure BDA0002257049310000031
(f denotes the feature matrix of the input image, g denotes the output matrix, H denotes the parameter matrix of the tracker, F, G, H corresponds to its frequency domain value, H*Conjugate complex number corresponding to H)
(the convolution operator has no practical physical meaning and can be regarded as a bridge for transferring the spatial domain correlation filtering to the frequency domain so as to improve the running speed of the computer)
The required filter is obtained in MOSSE by minimizing the sum of the squared errors between the convolved actual output and the convolved desired output, i.e. solving:
minH*∑i|Fi·H*-Gi|2(4)
(where i denotes the ith training sample)
To optimize the calculation, equation (4) is converted into:
minH*∑i|Fiwv·H* wv-Giwv|2(5)
(wherein the subscript denotes the w rows and v columns of the ith sample)
Unfolding to obtain:
minH*∑i[FiωvFiωv*(H* ωv)2-GiωvF* iωvHωv-FiωvH* ωvG* iωv+ GiωvG* iωv](6)
within the sum number is a sum of H*Is a solution of a first derivative equal to 0, which is H*Minimum value of (d):
Figure BDA0002257049310000032
the calculation result is as follows:
∑i[FiωvFiωv *Hωv-FiωvG* iωv]＝0 (8)
finally, the following steps are obtained:
Figure BDA0002257049310000033
the final closed-form solution of H is:
Figure BDA0002257049310000034
in consideration of the problems of preventing filter overfitting and enabling the filter to adapt to rotation, shielding, scale change and the like quickly, a parameter η of learning efficiency is introduced to represent the weights of frames with different time sequences, and the learning efficiency parameter enables the weight occupied by the frame with the time sequence closer to the current frame to be larger, and the learning result of the previous frame is exponentially decreased with time.
In order to make the filter have better adaptability to the external influences such as deformation, illumination and the like, an updating template is introduced by combining the parameters of learning efficiency:
Figure BDA0002257049310000041
Ai＝ηGi·Fi *+(1-η)Ai-1(12)
Bi＝ηFi·Fi *+(1-η)Bi-1(13)
(the learning rate η is 1 during initialization, and the variation range of η is 0-1 along with the variation of time sequence, wherein the subscripts i and i-1 respectively represent the numerator of the current frame and the last frame, A and B represent the numerator and denominator of the model formula of the filter)
After the maximum response of the target is obtained by the method, the corresponding maximum response is extracted as a new target coordinate, and the updated position coordinate X is respectively extracted1And Y1Comparing the obtained value with a central coordinate (300, 100), setting the value of X between 290 and 310 as a transverse shaking interval, namely when the final value of X of the selected target is in the interval, regarding the target as the transverse shaking of the mechanical arm, and sending 0X11 to a lower computer; x1When the average value is more than 310, sending 0X10 to a lower computer; when X is present1If the current time is less than 290, 0X09 is sent to the lower computer. Setting the y value between 90 and 110 as a longitudinal shaking interval, namely when the final y value of the selected target is in the interval, regarding the y value as the longitudinal shaking of the longitudinal arm, and sending 0X08 to the lower computer; x1When the current time is more than 110, sending 0X07 to the lower computer; when X is present1If the number is less than 110, 0X06 is sent to the lower computer.
The spatial positions of the three motors are given, and the motor 1 is a telescopic motor of the mechanical arm and is used for controlling the extension and retraction of the mechanical arm; the motor 2 is a pitching motor of the mechanical arm and controls the lifting and sinking of the mechanical arm; the motor 3 is a rotary motor of the mechanical arm and is used for adjusting the left-right swing of the mechanical arm, and the adjustment of the spatial position of the mechanical arm is completed by driving the three motors, so that the following function is realized.
The lower computer receives the instruction of the upper computer through the wireless transmission device and then runs the motor function to drive the corresponding motor. The servo motor file receives an instruction of an upper computer through the single chip microcomputer, writes an MODBUS protocol into a register according to bits through a write driver function, sends the MODBUS protocol to a driver through a serial port, and the host sets driver parameters and controls operation through the read-write register function of the MODBUS. The function codes supported by the driver are 0x3 (read register), 0x6 (write register), 0x78 (write target position), 0x7a (modify device address).
Drawings
Fig. 1 is an overall work flow diagram of the system.
FIG. 2 is a flow chart of the upper computer recognizing vision recognition and following
Fig. 3 is the spatial position of the robot arm motor.
Fig. 4 is a motor driving flowchart.
Detailed Description
The present invention is described in detail with reference to the embodiments shown in the drawings, but it should be understood that these embodiments are not intended to limit the present invention, and those skilled in the art should understand that functional, methodological, or structural equivalents or substitutions made by these embodiments are within the scope of the present invention.
Referring to fig. 1 to 4, fig. 1 is a flowchart illustrating the overall operation of the system, fig. 2 is a flowchart illustrating the recognition, visual recognition and tracking of the upper computer, fig. 3 is a spatial position of a robot motor, and fig. 4 is a flowchart illustrating the driving of the motor.
As shown in FIG. 1, the vision following system of the three-drive mechanical arm provided by the invention comprises three parts, namely an upper computer vision identification part, an upper computer and lower computer communication part and a lower computer drive control part. The upper computer visually identifies a frequency domain filtering mode based on convolution, predicts the position of a target possibly appearing in the next frame, mainly applies a minimum output mean square error and an MOSSE algorithm, extracts the position change coordinate of the target and sends the position change coordinate to the lower computer; the communication part of the upper computer and the lower computer is realized by serial communication, and data is transmitted according to bits; the lower computer wirelessly receives the command of the upper computer through serial port communication and sends a specified hexadecimal code to the driver through RS485 serial port communication so as to drive the corresponding motor, and the following action of the mechanical arm is completed through combination. The motor 1 is a telescopic motor of the mechanical arm and is used for controlling the extension and retraction of the mechanical arm; the motor 2 is a pitching motor of the mechanical arm and controls the lifting and sinking of the mechanical arm; the motor 3 is a rotary motor of the mechanical arm and is used for adjusting the left-right swing of the mechanical arm, the invention does not specially summarize the modes of the mechanical arm expansion, pitching and horizontal rotation, and the invention only provides a following system for driving three corresponding motors.
As shown in fig. 2, after a video image is acquired by a camera, a target to be tracked is selected from the video, then gaussian filtering is performed on a first frame acquired after selection, the image and a filter are mapped onto a topological structure, a boundary is filled in a cyclic image mode, namely, the left edge of the image is connected to the right edge, the top of the image is connected to the bottom of the image, and finally, point-by-point cosine window processing is adopted to enable the edge of the image to become zero.
In order to convert the spatial domain correlation filtering to the frequency domain filtering to improve the operation speed, a convolution operator is introduced, wherein when the conversion of the point product of the time domain convolution and the frequency domain is carried out:
g＝f*h (1)
G＝F·H*(2)
Figure BDA0002257049310000061
(f denotes the feature matrix of the input image, g denotes the output matrix, H denotes the parameter matrix of the tracker, F, G, H corresponds to its frequency domain value, H*Conjugate complex number corresponding to H)
(the convolution operator has no practical physical meaning and can be regarded as a bridge for transferring the spatial domain correlation filtering to the frequency domain so as to improve the running speed of the computer)
The required filter is obtained in MOSSE by minimizing the sum of the squared errors between the convolved actual output and the convolved desired output, i.e. solving:
minH*∑i|Fi·H*-Gi|2(4)
(where i denotes the ith training sample)
To optimize the calculation, equation (4) is converted into:
minH*∑i|Fiwv·H* wv-Giwv|2(5)
(wherein the subscript denotes the w rows and v columns of the ith sample)
Unfolding to obtain:
minH*∑i[FiωvFiωv *(H* ωv)2-GiωγF* iωγHωγ-FiωvH* ωvG* iωv+ GiωvG* iωv](6)
within the sum number is a sum of H*Is a solution of a first derivative equal to 0, which is H*Minimum value of (d):
Figure BDA0002257049310000062
the calculation result is as follows:
∑i[FiωvFiωv *Hωv-FiωvG* iωv]＝0 (8)
finally, the following steps are obtained:
Figure BDA0002257049310000071
the final closed-form solution of H is:
Figure BDA0002257049310000072
in consideration of preventing filter overfitting and enabling the filter to adapt to the problems of rotation, shielding, scale change and the like quickly, a parameter η of learning efficiency is introduced to represent the weights of frames with different time sequences, the learning efficiency parameter enables the weight occupied by the frame with the time sequence closer to the current frame to be larger, and the learning result of the previous frame is exponentially decreased gradually along with time, in order to enable the filter to have better adaptability to external influences such as deformation, illumination and the like, an updating template is introduced by combining the parameter of the learning efficiency:
Figure BDA0002257049310000073
Ai＝ηGi·Fi *+(1-η)Ai-1(12)
Bi＝ηFi·Fi *+(1-η)Bi-1(13)
(the learning rate η is 1 during initialization, and the variation range of η is 0-1 along with the variation of time sequence, wherein the subscripts i and i-1 respectively represent the numerator of the current frame and the last frame, A and B represent the numerator and denominator of the model formula of the filter)
After the maximum response of the target is obtained by the method, the corresponding maximum response is extracted as a new target coordinate, and the updated position coordinate X is respectively extracted1And Y1Comparing the obtained value with a central coordinate (300, 100), setting the value of X between 290 and 310 as a transverse shaking interval, namely when the final value of X of the selected target is in the interval, regarding the target as the transverse shaking of the mechanical arm, and sending 0X11 to a lower computer; x1When the average value is more than 310, sending 0X10 to a lower computer; when X is present1If the current time is less than 290, 0X09 is sent to the lower computer. Setting the y value between 90 and 110 as a longitudinal shaking interval, namely when the final y value of the selected target is in the interval, regarding the y value as the longitudinal shaking of the longitudinal arm, and sending 0X08 to the lower computer; x1When the current time is more than 110, sending 0X07 to the lower computer; when X is present1If < 110, sending 0X06 to the lower computer
As shown in fig. 3, the spatial positions of the three motors are given, and the motor 1 is a telescopic motor of the robot arm and is used for controlling the extension and retraction of the robot arm; the motor 2 is a pitching motor of the mechanical arm and controls the lifting and sinking of the mechanical arm; the motor 3 is a rotary motor of the mechanical arm and is used for adjusting the left-right swing of the mechanical arm, and the adjustment of the spatial position of the mechanical arm is completed by driving the three motors, so that the following function is realized.
As shown in fig. 4, the lower computer receives the instruction from the upper computer through the wireless transmission device and then runs the motor function to drive the corresponding motor. The servo motor file receives an instruction of an upper computer through the single chip microcomputer, writes an MODBUS protocol into a register according to bits through a write driver function, sends the MODBUS protocol to a driver through a serial port, and the host sets driver parameters and controls operation through the read-write register function of the MODBUS. The function codes supported by the driver are 0x3 (read register), 0x6 (write register), 0x78 (write target position), 0x7a (modify device address). The driver control motor speed mode comprises the following specific steps:
(1) write device 1 (function code 0x06 address 0MODBUS Enable) 1
(2) Reading equipment 1 (function code 0x06 address 14 alarm code) (when reading 0, it is normal)
(3) Writing device 1 (function code 0x06 address 3 motor acceleration) 1
(4) Writing device 1 (function code 0x06 address 2 motor target speed) 1
(5) And (4) repeating the process to control the rotating speed of the motor.
The received upper computer instruction corresponds to the mechanical arm action and the motor rotation condition as follows:
0x 06: the constant speed is positive rotation towards the No. 2 motor at the constant speed;
0x 07: the motor No. 2 is turned upwards at a constant speed and is rotated reversely at a constant speed;
0x 08: stopping the motor No. 2, locking and stopping;
0x 09: the constant speed positive rotation is carried out on the motor with the constant speed being right No. 3;
0x 10: the constant speed is reversed to the left No. 3 motor at a constant speed;
0x 11: and stopping the No. 3 motor, locking and stopping.
The vision following system of the three-drive mechanical arm provided by the invention has the following beneficial effects:
(1) the machine vision is applied to the field of mechanical arms, and the tracking of the mechanical arms on the selected targets is realized by selecting the targets in the video, so that the tracking of the mechanical arms is more efficient and accurate.
(2) The creative division into six regions in the visual follow is the three divided regions in the X-axis and the three divided regions in the Y-axis explained in detail in the present invention.
(3) The vision following system of the three-drive mechanical arm has very good universality, can run on various upper computers (linux systems, raspberry pies and the like), and is a mature product in the market, and the three driven motors are six-wire servo motors.
The above-listed detailed description is only a specific description of a possible embodiment of the present invention, and they are not intended to limit the scope of the present invention, and equivalent embodiments or modifications made without departing from the technical spirit of the present invention should be included in the scope of the present invention.
It will be evident to those skilled in the art that the invention is not limited to the details of the foregoing illustrative embodiments, and that the present invention may be embodied in other specific forms without departing from the spirit or essential attributes thereof. The present embodiments are therefore to be considered in all respects as illustrative and not restrictive, the scope of the invention being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein.

Claims (7)

1. A vision following system of a three-drive mechanical arm is characterized by comprising three parts, namely an upper computer vision identification part, an upper computer and lower computer communication part and a lower computer drive control part:
(1) the upper computer vision recognition system of the system adopts the idea of relevant filtering, processes the image collected by the camera by applying the convolution theorem, finds the target function of the MOSSE filter, solves the closed solution of the MOSSE filter, and updates the peak coordinate to realize target tracking, thereby forming a vision recognition program and respectively extracting the updated position coordinate X1And Y1Comparing the central coordinates with the central coordinates (300, 100), obtaining the result, outputting the result to a lower computer, and obtaining a computerThe motion trail of the mechanical arm;
(2) for the motor drive of a lower computer, a register is configured in a microprocessor to compile a read-write driver function, the communication between a driver and the microprocessor is completed through an RS485 serial port, and the current position information of the mechanical arm is fed back to the microprocessor in real time through an encoder; the read-write driver function achieves the purpose of control by changing the state of the flag bit in the register; the specific implementation is that after receiving the instruction of the upper computer, the MODBUS protocol is written into the register by writing the function of the driver according to the bit, and is sent to the driver through the serial port, and the host computer sets the parameter of the driver and controls the operation through the read-write register function of the MODBUS.
2. The visual following system of a three-drive robotic arm of claim 1, wherein: after a camera acquires a video image, selecting a target to be tracked in the video, then performing Gaussian filtering on a first frame acquired after selection, mapping the image and a filter onto a topological structure, filling a boundary in a circulating image mode, namely connecting the left edge of the image to the right edge, connecting the top to the bottom, and finally performing point-by-point cosine window processing to enable the edge of the image to become zero.
3. The visual following system of a three-drive robotic arm of claim 1, wherein: the i-th frame image is processed in MOSSE by minimizing the sum of the squared errors between the convolved actual output and the convolved desired output to obtain the required filter.
4. The visual following system of claim 1, wherein a learning efficiency parameter η is introduced to indicate the weights of frames with different time sequences, the learning efficiency parameter is such that the closer the time sequence is to the current frame, the higher the weight of the frame is, the learning result of the previous frame decreases exponentially with time, and the change range of η is 0-1 with the change of the time sequence.
5. According to claimThe vision following system of claim 1, wherein: after the maximum response of the target is obtained, extracting the corresponding maximum response as a new target coordinate, and respectively extracting the updated position coordinate X1And Y1Comparing the obtained value with a central coordinate (300, 100), setting the value of X between 290 and 310 as a transverse shaking interval, namely when the final value of X of the selected target is in the interval, regarding the target as the transverse shaking of the mechanical arm, and sending 0X11 to a lower computer; x1>Sending 0X10 to the lower computer at 310; when X is present1<When 290, sending 0X09 to the lower computer;
setting the y value between 90 and 110 as a longitudinal shaking interval, namely when the final y value of the selected target is in the interval, regarding the y value as the longitudinal shaking of the longitudinal arm, and sending 0X08 to the lower computer; y is1>When 110, sending 0X07 to the lower computer; when Y is1<When 110, sending 0X06 to the lower computer;
the spatial positions of the three motors are given, and the first motor is a telescopic motor of the mechanical arm and is used for controlling the extension and retraction of the mechanical arm; the second motor is a pitching motor of the mechanical arm and controls the lifting and sinking of the mechanical arm; the third motor is a rotary motor of the mechanical arm and is used for adjusting the left-right swing of the mechanical arm, and the adjustment of the spatial position of the mechanical arm is completed by driving the three motors, so that the following function is realized;
0x 06: the constant speed is positive rotation towards the No. 2 motor at the constant speed;
0x 07: the motor No. 2 is turned upwards at a constant speed and is rotated reversely at a constant speed;
0x 08: stopping the motor No. 2, locking and stopping;
0x 09: the constant speed positive rotation is carried out on the motor with the constant speed being right No. 3;
0x 10: the constant speed is reversed to the left No. 3 motor at a constant speed;
0x 11: and stopping the No. 3 motor, locking and stopping.
6. The visual following system of a three-drive robotic arm of claim 1, wherein: determining a target is to find a filter h that maximizes its response at the target; f represents a training image, g represents an output image, h represents a filter, and F, G, H respectively correspond to frequency domain values thereof; deducing a calculated value of the target H through a time domain formula and a frequency domain formula to complete the conversion of the time domain convolution and the frequency domain convolution; solving a closed solution of H by using a target function of the filter and updating the MOSSE; calling a python corresponding library function to generate an ideal response of a Gaussian shape, solving a target frame area and performing filter convolution to obtain a response value, extracting a corresponding maximum value as a new target position, updating a position coordinate new target as a center selection target, finally respectively extracting updated X and Y values, comparing and sending the updated X and Y values to a lower computer; the execution device of the lower computer is a servo motor, the servo motor selects a speed mode, and the three motors respectively control different polar coordinate parameters to realize space motion; the upper computer converts the comparison result of the object, namely the deviation degree of the target relative to the center of the visual field, into the specific displacement direction and the movement time of the lower computer, so that the mechanical arm can adjust the position and the posture along with the movement of the target.
7. The visual following system of a three-drive robotic arm of claim 1, wherein: after a camera acquires a video image, selecting a target to be tracked in the video, then performing Gaussian filtering on a first frame acquired after selection, mapping the image and a filter onto a topological structure, filling a boundary in a circulating image mode, namely connecting the left edge of the image to the right edge, connecting the top to the bottom, and finally performing point-by-point cosine window processing to enable the edge of the image to become zero;
in order to convert the spatial domain correlation filtering to the frequency domain filtering to improve the operation speed, a convolution operator is introduced, wherein when the conversion of the point product of the time domain convolution and the frequency domain is carried out:
g＝f*h (1)
G＝F·H*(2)
Figure FDA0002257049300000031
where f denotes the feature matrix of the input image, g denotes the output matrix, h denotes the parameter matrix of the tracker, F, G, H corresponds to its frequency domain values，H*A complex conjugate corresponding to H;
the required filter is obtained in MOSSE by minimizing the sum of the squared errors between the convolved actual output and the convolved desired output, i.e. solving:
minH*∑i|Fi·H*-Gi|2(4)
wherein i represents the ith training sample;
to optimize the calculation, equation (4) is converted into:
Figure FDA0002257049300000033
where the subscripts denote the w rows and v columns of the ith sample;
unfolding to obtain:
Figure FDA0002257049300000034
within the sum number is a sum of H*Is a solution of a first derivative equal to 0, which is H*Minimum value of (d):
Figure FDA0002257049300000032
the calculation result is as follows:
∑i[FiωvFiωv *Hωv-FiωvG* iωv]＝0 (8)
finally, the following steps are obtained:
Figure FDA0002257049300000041
the final closed-form solution of H is:
Figure FDA0002257049300000042
in consideration of preventing filter overfitting, a parameter η of learning efficiency is introduced to represent the weights of frames with different time sequences, the learning efficiency parameter enables the weight occupied by the frame with the time sequence closer to the current frame to be larger, and the learning result of the previous frame is exponentially decreased along with the time;
in order to make the filter have better adaptability to the external influence, an updating template is introduced by combining parameters of learning efficiency:
Figure FDA0002257049300000043
Ai＝ηGi·Fi *+(1-η)Ai-1(12)
Bi＝ηFi·Fi *+(1-η)Bi-1(13)
in the initialization process, the learning rate η is 1, and the change range of η is 0-1 along with the change of time sequence, wherein lower subscripts i and i-1 respectively represent the numerator of a current frame and the numerator of a previous frame, and A and B represent the numerator and the denominator of a model formula of the filter;
after the maximum response of the target is obtained by the method, the corresponding maximum response is extracted as a new target coordinate, and the updated position coordinate X is respectively extracted1And Y1Comparing the obtained value with a central coordinate (300, 100), setting the value of X between 290 and 310 as a transverse shaking interval, namely when the final value of X of the selected target is in the interval, regarding the target as the transverse shaking of the mechanical arm, and sending 0X11 to a lower computer; x1>Sending 0X10 to the lower computer at 310; when X is present1<When 290, sending 0X09 to the lower computer; setting the y value between 90 and 110 as a longitudinal shaking interval, namely when the final y value of the selected target is in the interval, regarding the y value as the longitudinal shaking of the longitudinal arm, and sending 0X08 to the lower computer; x1>When 110, sending 0X07 to the lower computer; when X is present1<When 110, sending 0X06 to the lower computer;
the spatial positions of the three motors are given, and the motor 1 is a telescopic motor of the mechanical arm and is used for controlling the extension and retraction of the mechanical arm; the motor 2 is a pitching motor of the mechanical arm and controls the lifting and sinking of the mechanical arm; the motor 3 is a rotary motor of the mechanical arm and is used for adjusting the left-right swing of the mechanical arm, and the adjustment of the spatial position of the mechanical arm is completed by driving the three motors, so that the following function is realized;
the lower computer receives the instruction of the upper computer through the wireless transmission device and then runs a motor function to drive a corresponding motor; the servo motor file receives an instruction of an upper computer through the single chip microcomputer, writes an MODBUS protocol into a register according to bits through a write driver function, and sends the MODBUS protocol to a driver through a serial port, and the host sets driver parameters and controls operation through the read-write register function of MODBUS; the function codes supported by the driver are 0x3 (read register), 0x6 (write register), 0x78 (write target position), 0x7a (modify device address).
CN201911057970.4A 2019-11-01 2019-11-01 Visual following system of three drive arms Pending CN110977960A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201911057970.4A CN110977960A (en) 2019-11-01 2019-11-01 Visual following system of three drive arms

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201911057970.4A CN110977960A (en) 2019-11-01 2019-11-01 Visual following system of three drive arms

Publications (1)

Publication Number Publication Date
CN110977960A true CN110977960A (en) 2020-04-10

Family

ID=70082857

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201911057970.4A Pending CN110977960A (en) 2019-11-01 2019-11-01 Visual following system of three drive arms

Country Status (1)

Country Link
CN (1) CN110977960A (en)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE3405909A1 (en) * 1984-02-18 1985-08-22 Licentia Patent-Verwaltungs-Gmbh, 6000 Frankfurt DEVICE FOR DETECTING, MEASURING ANALYSIS AND / OR REGULATING TECHNICAL PROCEDURES
CN106426279A (en) * 2016-11-15 2017-02-22 北京工业大学 Telescopic mechanical arm
CN107363837A (en) * 2017-09-08 2017-11-21 桂林加宏汽车修理有限公司 A kind of method of manipulator control system and control machinery hand captures object
CN108890656A (en) * 2018-07-11 2018-11-27 北京工业大学 A kind of telescopic service robot of the wheel drive containing friction
CN109177175A (en) * 2018-07-10 2019-01-11 泉州装备制造研究所 A kind of 3D printing spray head end real-time tracking localization method

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE3405909A1 (en) * 1984-02-18 1985-08-22 Licentia Patent-Verwaltungs-Gmbh, 6000 Frankfurt DEVICE FOR DETECTING, MEASURING ANALYSIS AND / OR REGULATING TECHNICAL PROCEDURES
CN106426279A (en) * 2016-11-15 2017-02-22 北京工业大学 Telescopic mechanical arm
CN107363837A (en) * 2017-09-08 2017-11-21 桂林加宏汽车修理有限公司 A kind of method of manipulator control system and control machinery hand captures object
CN109177175A (en) * 2018-07-10 2019-01-11 泉州装备制造研究所 A kind of 3D printing spray head end real-time tracking localization method
CN108890656A (en) * 2018-07-11 2018-11-27 北京工业大学 A kind of telescopic service robot of the wheel drive containing friction

Non-Patent Citations (5)

* Cited by examiner, † Cited by third party
Title
DAVID S.BOLME， J.ROSS BEVERIDGE，BRUCE A.DRAPER，YUI MAN LUI: "Visual object tracking using adaptive correlation filters", 《2010 IEEE COMPUTER SOCIETY CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION》 *
么跃轩: "汽车电动座椅调整机械手的视觉跟踪与机器学习", 《中国优秀硕士学位论文全文数据库》 *
刘海成: "《AVR单片机原理及测控工程应用-基于ATmega48/ATmega16》", 30 June 2015 *
夏菠: "基于双目视觉的动态目标定位与抓取研究", 《中国优秀硕士学位论文全文数据库》 *
王西彬，焦黎，周天丰: "《精密制造工学基础》", 31 January 2018 *

Similar Documents

Publication Publication Date Title
CN112132894B (en) Mechanical arm real-time tracking method based on binocular vision guidance
CN107901041B (en) Robot vision servo control method based on image mixing moment
CN111775152B (en) Method and system for guiding mechanical arm to grab scattered stacked workpieces based on three-dimensional measurement
CN111360827B (en) Visual servo switching control method and system
CN103325106B (en) Based on the Moving Workpieces method for sorting of LabVIEW
CN111462154B (en) Target positioning method and device based on depth vision sensor and automatic grabbing robot
CN111872934B (en) Mechanical arm control method and system based on hidden semi-Markov model
CN113771045B (en) Vision-guided high-adaptability positioning and grabbing method for middle frame of right-angle robot mobile phone
CN110605721A (en) Mechanical arm dragging teaching method based on terminal six-dimensional force sensor
CN105014677A (en) Visual mechanical arm control device and method based on Camshift visual tracking and D-H modeling algorithms
CN110977990A (en) Mechanical arm dragging teaching method based on terminal six-dimensional force sensor
CN113706621B (en) Mark point positioning and posture obtaining method and system based on marked image
CN112589232A (en) Weld joint tracking method and device based on independent deviation correction type deep learning
Husain et al. Realtime tracking and grasping of a moving object from range video
CN109465829A (en) A kind of industrial robot geometric parameter discrimination method based on transition matrix error model
CN113400301B (en) Robot 3D hand-eye calibration method, system, device and medium
CN113172632A (en) Simplified robot vision servo control method based on images
CN111993422A (en) Robot axis and hole alignment control method based on uncalibrated vision
CN113618367B (en) Multi-vision space assembly system based on seven-degree-of-freedom parallel double-module robot
CN113910218B (en) Robot calibration method and device based on kinematic and deep neural network fusion
CN111331604A (en) Machine vision-based valve screwing flexible operation method
CN116872216B (en) Robot vision servo operation method based on finite time control
CN110977960A (en) Visual following system of three drive arms
CN117340929A (en) Flexible clamping jaw grabbing and disposing device and method based on three-dimensional point cloud data
CN114434441A (en) Mobile robot visual servo tracking control method based on self-adaptive dynamic programming

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
WD01 Invention patent application deemed withdrawn after publication
WD01 Invention patent application deemed withdrawn after publication

Application publication date: 20200410