CN103900497B - Based on the contactless digger operating device attitude measurement method of vision measurement - Google Patents

Abstract

A non-contact excavator working device attitude measurement method based on vision measurement, which pastes a circular piece with obvious image saddle point features on the working device as a marked feature point, and then uses the industrial camera on the cab frame to capture The image of the working device of the excavator, and then apply the saddle point detection method to detect all the image saddle points on the working device image including the center point of the feature sheet in real time, and filter out the saddle points of the image that are not the center point of the feature sheet through the distance between the feature sheets. Furthermore, the inclination angles of the components of the working device are obtained through the inclination angles of the lines between the center points of the feature sheets, so as to measure the attitude of the working device. The measuring method can dynamically measure the attitude of the working device of the excavator, and does not install any vulnerable and precise parts on the working device, so that the reliability of the excavator's measurement and control is improved and the maintenance cost is reduced. Moreover, the detection result is accurate, the calculation steps are few, the method is simple, the dynamic response speed of the measurement is fast, and the real-time performance is good.

Description

Attitude measurement method of non-contact excavator working device based on vision measurement

technical field

The invention relates to a visual measurement method, in particular to a visual measurement-based posture measurement method for excavator working devices.

technical background

The hydraulic excavator is one of the construction machinery with the most typical functions, the most complex structure and the widest range of uses. As a mainstream product of construction machinery, it plays an extremely important role in the construction of industrial and civil buildings, transportation, water conservancy and electric power engineering, mining and military engineering.

A typical single-bucket hydraulic excavator consists of three parts: working device, slewing platform and walking device. The working device is a component that directly completes the excavation task. It is composed of three parts: the boom, the stick, and the bucket. Boom hydraulic cylinder, stick hydraulic cylinder, bucket hydraulic cylinder. The operation of the excavator is: through the joint operation control of the hydraulic motor of the slewing platform and the three hydraulic cylinders, the excavator bucket is positioned at the excavation point, and then through the joint operation of the bucket hydraulic cylinder and the stick hydraulic cylinder. For excavation or planing, after the bucket is filled with materials, the bucket is turned to the unloading point through the joint operation of the hydraulic motor of the slewing platform and the three hydraulic cylinders to perform the unloading action, and finally the bucket turns again Go to the digging point for the next digging operation. In short, the operation process of the excavator is complicated, and usually the hydraulic motor of the slewing platform and the boom hydraulic cylinder, stick hydraulic cylinder, and bucket hydraulic cylinder need to be jointly operated at the same time, and it is necessary to judge the boom, stick , The position and angle of the bucket. It can be seen that excavators have high technical requirements for operators, and they need professional training before they can work, and their labor intensity is high, and the working environment is harsh; it poses serious constraints to the construction efficiency, cost and quality of various projects, and it is urgent to realize The operation automation of the excavator solves the above problems.

In recent decades, some major construction machinery enterprises and universities at home and abroad have done a lot of research work on the automation of excavator operations. For example, foreign countries include Komatsu Corporation of Japan, Case Corporation of the United States, and Lancaster University of the United Kingdom; domestic companies include Tongji University, Zhejiang University, and Sunward Intelligent Corporation. The developed automatic excavator: the operator only needs to issue instructions, give the excavation point and its excavation task (type), and the control system controls the hydraulic motor and The actions of the three hydraulic cylinders automatically realize the corresponding excavation operations. Its body positioning information is usually completed by the GPS navigation system. The acquisition of the attitude information of the working device is mostly achieved by installing an inclination sensor or a photoelectric encoder at the hinge points of the boom, stick and bucket of the working device. The problem with the attitude information acquisition method is that during the digging operation of the excavator, the working device will inevitably collide with the soil, rocks, or other objects, resulting in violent vibrations, which can easily cause damage to the sensors installed on it, and eventually As a result, the entire automation control system cannot operate normally. The reliability of the system is low and the maintenance cost is high.

Contents of the invention

The present invention aims at the serious deficiency of the existing automated excavator working device attitude measurement system, and proposes a non-contact excavator working device attitude measurement method, which can dynamically measure the attitude information of the excavator working device. No vulnerable and precise parts are installed on the working device, which improves the reliability of the excavator's measurement and control and reduces the maintenance cost. Moreover, the detection result is accurate, the calculation steps are few, the method is simple, the dynamic response speed of the measurement is fast, and the real-time performance is good.

The technical solution adopted by the present invention to solve the technical problem is: a non-contact excavator working device posture measurement method based on visual measurement, comprising the following steps:

A. Arrangement of cameras and feature sheets:

Install the industrial camera on the excavator cab frame, the field of view of the industrial camera covers the working device of the excavator from one side, and the USB interface of the industrial camera is connected to the automatic control system of the excavator;

At the same time, paste two circular feature sheets on the side of the excavator arm, stick and bucket facing the industrial camera. The surface of the feature sheet is divided into four 90-degree fan-shaped areas, of which the two opposite fan-shaped areas are black area, and the other two relative fan-shaped areas are white areas, so that the center point of the feature sheet can be detected as a saddle point in the saddle point detection of the image;

The line connecting the center points of the two characteristic pieces on the boom is parallel to the line connecting the hinge points of the two rods on the boom; The line connecting the points is parallel; the line connecting the center points of the two feature pieces on the bucket is parallel to the line connecting the hinge point of the rod of the bucket and the tip of the shovel;

The distance between the center points of the two feature plates on the boom is l ₁ , the distance between the center points of the two feature plates on the arm is l ₂ , and the distance between the center points of the two feature plates on the bucket is l ₃ , and l ₁ ≠l ₂ ≠l ₃ , l ₁ ≠l ₃ ;

B. Image acquisition: When the excavator is working, the industrial camera captures a digital image every 60-80ms, and transmits the digital image to the automatic control system through the USB interface;

C. Saddle point detection:

During the initial detection, the automatic control system detects the saddle point i in the entire digital image through the saddle point search algorithm, and obtains the image coordinate system coordinates P' _i (r _i , c _i ) of the saddle point i, where i is the serial number of the saddle point, i= 1,2,...n, r _i is the row coordinate of the saddle point, and _ci is the column coordinate of the saddle point;

After the first detection, the automatic control system detects the saddle point i in the rectangular detection area determined by the step F of the previous detection through the saddle point search algorithm, and obtains the image coordinate system coordinates P' _i (r _i , c _i ) of the saddle point i, Wherein, i is the sequence number of the saddle point, i=1,2,...n, r _i is the row coordinate of the saddle point, and c _i is the column coordinate of the saddle point;

D. Coordinate transformation: convert the image coordinate system coordinates P' _i (r _i , c _i ) of the saddle point i into an xy plane with the photographed sides of the boom (1), stick (2) and bucket (3) The coordinates P _i (x _i y _i ,0) of the world coordinate system (WCS);

E. Saddle point filtering and inclination measurement: calculate the mutual distance between any two saddle points in the world coordinate system, that is, the saddle point distance a _jk , and obtain the distance matrix L:

In the formula: a _jk represents the distance between the i=j saddle point and the i=k saddle point, and L is a symmetric matrix;

Calculate the absolute value |a _jk -l ₁ | of the difference between all saddle point distances a _jk and the saddle point distance l ₁ of the two feature pieces on the boom, if the value of |a _j1,k1 -l ₁ | is the smallest, then determine The corresponding two saddle points j1 and k1 are the center points of the two feature sheets on the boom; the inclination angle of the line connecting the two center points is α ₁ :

Calculate the absolute value |a _jk -l ₂ | of the difference between all saddle point distances a _jk and the saddle point distance l ₂ of the two feature pieces on the stick, if the value of |a _j2,k2 -l ₂ | is the smallest, then determine The corresponding two saddle points j2 and k2 are the center points of the two feature sheets on the boom; the inclination angle of the line connecting the two center points is α ₂ :

Calculate the absolute value |a _jk -l ₃ | of the difference between all saddle point distances a _jk and the saddle point distance l ₃ of the two feature pieces on the bucket, if the value of |a _j3,k3 -l ₃ | is the smallest, then determine The corresponding two saddle points j3 and k3 are the center points of the two feature sheets on the boom; the inclination angle of the line connecting the two center points is α ₃ :

F. Narrow down the detection area: Find the leftmost center point coordinate L(x _l ,y _l ,0) and the rightmost center point coordinate R in the world coordinate system coordinates of the center point of the six currently detected feature sheets (x _r ,y _r ,0), the uppermost central point coordinate T(x _t ,y _t ,0) and the lowermost central point coordinate B(x _b ,y _b ,0), x ₀ is all the set The maximum distance of the lateral movement of the feature sheet in the detection time interval, _y0 is the maximum distance of the longitudinal movement of all the feature sheets in the detection time interval; then the coordinates of the two diagonal points of the rectangular area to be detected next time are: LB(x _l -x ₀ ,y _l -y ₀ ,0), TR(x _r +x ₀ ,y _r +y ₀ ,0), and then through coordinate inverse transformation, the two opposite corners of the next rectangular detection area can be obtained The coordinates of the image coordinate system, and then determine the rectangular detection area in the image coordinate system for the next detection;

G. Repeat steps B-F to dynamically measure the posture of the excavator working device in real time.

Compared with prior art, the benefit effect of the present invention is:

1. The present invention proposes a non-contact excavator working device posture measurement method based on visual measurement technology. In this method, the industrial camera is installed on the excavator cab frame, and the working device is pasted with cheap, non-destructive The colored circular piece is used as the marking feature point, which has a simple structure and low purchase cost; at the same time, it avoids the installation of precision measuring parts such as inclination sensors on the working device with large movements, easy collision with the work object, and strong vibration, which may be easily damaged. The defect improves the reliability of the excavator measurement and control system and reduces the maintenance cost.

2. By pasting a circular piece with obvious image saddle point features on the working device as a marked feature point, and then applying the saddle point detection method to the captured image of the excavator working device for online real-time processing, and detecting the central point of the feature sheet on the working device All the image saddle points in the image, and filter out the image saddle points of the center points of the non-feature slices through the distance between the feature slices, and then obtain the inclination angle of each part of the working device through the inclination angle of the line between the center points of the feature slices, so as to measure The attitude of the working device. Compared with the method of analyzing and capturing the contour line on a large area of the entire image, the detection result is accurate, the calculation steps are few, and the method is simple, which improves the dynamic response speed of the measurement system and ensures the real-time requirements. Subsequent detections after the initial detection are only detected in the area of feature points slightly larger than the previous detection, which further shortens the search time of saddle points and speeds up the measurement speed.

The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments.

Description of drawings

FIG. 1 is a schematic diagram of the hardware arrangement adopted in the embodiment of the present invention.

Fig. 2 is the rectangular area (the solid line area in the figure) where the 6 feature sheet center points detected this time obtained in step B of the embodiment of the present invention and the predicted rectangular detection area (the area in the figure) next time dashed area).

Fig. 3 is a comparison diagram of the result of the method of the present invention obtained through experiments and the boom inclination measured by installing the sensor on the hinge point of the boom.

detailed description

Example

Figure 1 shows that a specific embodiment of the present invention is: a non-contact working device attitude measurement method based on vision measurement, comprising the following steps:

A. Arrangement of cameras and feature sheets:

The industrial camera 4 is installed on the excavator cab frame 5, the field of view of the industrial camera 4 covers the working device of the excavator from one side, and the USB interface of the industrial camera 4 is connected with the automatic control system of the excavator;

At the same time, two circular feature sheets 6 are pasted on the sides of the excavator arm 1, arm 2 and bucket 3 facing the industrial camera 4, and the surface of the feature sheet 6 is divided into four 90-degree fan-shaped areas, wherein the opposite Two fan-shaped areas are black areas 6a, and the other two relative fan-shaped areas are white areas 6b, so that the center point of the feature sheet 6 can be detected as a saddle point in the saddle point detection of the image;

The line connecting the center points of the two feature pieces 6 on the boom 1 is parallel to the line connecting the hinge points of the two rods on the boom 1; the line connecting the center points of the two feature pieces 6 on the arm 2 is parallel to The connection line of the upper two hinge points of the rods is parallel; the connection line of the center points of the two feature pieces 6 on the bucket 3 is parallel to the connection line between the hinge points of the rods of the bucket 3 and the tip of the shovel;

In the present invention, the hinge points of the two rods on the boom 1 refer to the hinge points between the boom 1 and the slewing platform and the hinge points between the boom 1 and the stick 2; Point refers to the hinge point between arm 2 and boom 1 and the hinge point between arm 2 and bucket 3; the rod hinge point of bucket 3 refers to the hinge point between bucket 3 and arm 2.

The distance between the center points of the two feature pieces 6 on the boom 1 is l ₁ , the distance between the center points of the two feature pieces 6 on the arm 2 is l ₂ , and the distance between the two feature pieces 6 on the bucket 3 is The center point distance is l ₃ , and l ₁ ≠l ₂ ≠l ₃ , l ₁ ≠l ₃ ;

B. Image acquisition: when the excavator is working, the industrial camera 4 captures a digital image every 60-80ms, and transmits the digital image to the automatic control system through the USB interface;

C. Saddle point detection:

During the initial detection, the automatic control system detects the saddle point i in the entire digital image through the saddle point search algorithm, and obtains the image coordinate system coordinates P' _i (r _i , c _i ) of the saddle point i, where i is the serial number of the saddle point, i= 1,2,...n, r _i is the row coordinate of the saddle point, and _ci is the column coordinate of the saddle point;

After the first detection, the automatic control system detects the saddle point i in the rectangular detection area determined by the step F of the previous detection through the saddle point search algorithm, and obtains the image coordinate system coordinates P' _i (r _i , c _i ) of the saddle point i, Wherein, i is the sequence number of the saddle point, i=1,2,...n, r _i is the row coordinate of the saddle point, and c _i is the column coordinate of the saddle point;

D. Coordinate transformation: transform the image coordinate system coordinates P' _i (r _i , c _i ) of the saddle point i into the world coordinate system (WCS) with the xy plane as the photographed side of the boom 1, stick 2 and bucket 3 ) coordinates P _i ( _xi y _i ,0);

E. Saddle point filtering and inclination measurement: calculate the mutual distance between any two saddle points in the world coordinate system, that is, the saddle point distance a _jk , and obtain the distance matrix L:

In the formula: a _jk represents the distance between the i=j saddle point and the i=k saddle point, and L is a symmetric matrix;

Calculate the absolute value |a _jk -l ₁ | of the difference between all the saddle point distances a _jk and the saddle point distance l ₁ of the two feature pieces 6 on the boom 1, if the value of |a _j1,k1 -l ₁ | is the smallest, Then it is determined that the corresponding two saddle points j1 and k1 are the center points of the two feature pieces 6 on the boom; the inclination angle of the line connecting the two center points is α ₁ :

Calculate the absolute value |a _jk -l ₂ | of the difference between all the saddle point distances a _jk and the saddle point distance l ₂ of the two feature pieces 6 on the stick 2, if the value of |a _j2,k2 -l ₂ | is the smallest, Then it is determined that the corresponding two saddle points j2 and k2 are the center points of the two feature pieces 6 on the boom; the inclination angle of the line connecting the two center points is α ₂ :

Calculate the absolute value |a _jk -l ₃ | of the difference between all the saddle point distances a _jk and the saddle point distance l ₃ of the two feature pieces 6 on the bucket 3, if the value of |a _j3,k3 -l ₃ | is the smallest, Then it is determined that the corresponding two saddle points j3 and k3 are the center points of the two feature pieces 6 on the boom; the inclination angle of the line connecting the two center points is α ₃ :

F. Narrowing down the detection area: Referring to Fig. 2, in the world coordinate system coordinates of the center points of the six currently detected feature sheets 6, find the leftmost center point coordinate L(x _l , y _l , 0), the rightmost The center point coordinates R(x _r ,y _r ,0), the uppermost center point coordinates T(x _t ,y _t ,0) and the lowermost center point coordinates B(x _b ,y _b ,0), x ₀ is the maximum distance of lateral movement of all feature sheets 6 in the detection time interval, and _y0 is the maximum distance of longitudinal movement of all feature sheets in the detection time interval; It is: LB(x _l -x ₀ ,y _l -y ₀ ,0), TR(x _r +x ₀ ,y _r +y ₀ ,0), and then through the inverse transformation of the coordinates, the next rectangular detection area can be obtained The image coordinate system coordinates of the two diagonal points, and then determine the rectangular detection area in the image coordinate system for the next detection;

G. Repeat steps B-F to dynamically measure the posture of the excavator working device in real time.

Fig. 3 is a comparison diagram of the method of the present invention (l ₁ = 100mm, l ₂ = 200mm, l ₃ = 300mm) and the inclination results of the boom measured under static conditions when the sensor is installed on the hinge point of the boom. . During the experiment, the value of _l1 is 100mm, the value of _l2 is 200mm, and the value of l3 is 300mm. As can be seen from Figure ₃ , the relative measurement error between the two is within 0.04° under static conditions; The time used by the method is 60-80ms.

Experiments show that the method of the invention can meet the requirements of measurement accuracy and real-time performance under static conditions, and has feasibility and applicability in attitude measurement of excavator working devices.

Claims (1)

1. A non-contact working device attitude measurement method based on visual measurement, comprising the following steps: A. Arrangement of cameras and feature sheets: Install the industrial camera (4) on the excavator cab frame (5), the field of view of the industrial camera (4) covers the working device of the excavator from one side, the USB interface of the industrial camera (4) and the excavator connected to the automatic control system; At the same time, paste two circular feature sheets (6) on the sides of the excavator arm (1), stick (2) and bucket (3) facing the industrial camera (4), and the surface of the feature sheet (6) Divided into four 90-degree fan-shaped areas, wherein the two opposite fan-shaped areas are black areas (6a), and the other two relative fan-shaped areas are white areas (6b), so that the center point of the feature sheet (6) is at the saddle point of the image Can be detected as a saddle point in the detection; The line connecting the center points of the two feature pieces (6) on the boom (1) is parallel to the line connecting the hinge points of the two rods on the boom (1); the two feature pieces (6) on the stick (2) are ) is parallel to the line connecting the hinge points of the two rods on the bucket (2); the line connecting the center points of the two feature pieces (6) on the bucket (3) is parallel to the The connecting line between the hinge point of the rod and the tip of the shovel is parallel; The distance between the center points of the two feature pieces (6) on the boom (1) is l ₁ , the distance between the center points of the two feature pieces (6) on the stick (2) is l ₂ , and the bucket ( 3) The distance between the center points of the two feature sheets (6) is l ₃ , and l ₁ ≠l ₂ ≠l ₃ , l ₁ ≠l ₃ ; B. Image acquisition: when the excavator is working, the industrial camera (4) captures a digital image every 60-80ms, and transmits the digital image to the automatic control system through the USB interface; C. Saddle point detection: During the initial detection, the automatic control system detects the saddle point i in the entire digital image through the saddle point search algorithm, and obtains the image coordinate system coordinates P' _i (r _i , c _i ) of the saddle point i, where i is the serial number of the saddle point, i= 1,2,...n, r _i is the row coordinate of the saddle point, and _ci is the column coordinate of the saddle point; After the first detection, the automatic control system detects the saddle point i in the rectangular detection area determined by the step F of the previous detection through the saddle point search algorithm, and obtains the image coordinate system coordinates P' _i (r _i , c _i ) of the saddle point i, Wherein, i is the sequence number of the saddle point, i=1,2,...n, r _i is the row coordinate of the saddle point, and c _i is the column coordinate of the saddle point; D. Coordinate transformation: convert the image coordinate system coordinates P' _i (r _i , c _i ) of the saddle point i into an xy plane with the photographed sides of the boom (1), stick (2) and bucket (3) The coordinates P _i (x _i y _i ,0) of the world coordinate system (WCS); E. Saddle point filtering and inclination measurement: calculate the mutual distance between any two saddle points in the world coordinate system, that is, the saddle point distance a _jk , and obtain the distance matrix L: In the formula: a _jk represents the distance between the i=j saddle point and the i=k saddle point, and L is a symmetric matrix; Calculate the absolute value |a _jk -l ₁ | of the difference between all the saddle point distances a _jk and the saddle point distance l ₁ of the two feature pieces (6) on the boom (1), if |a _j1,k1 -l ₁ | value is the smallest, then it is determined that the corresponding two saddle points j1 and k1 are the center points of the two feature pieces (6) on the boom; the inclination angle of the line connecting the two center points is α ₁ : Calculate the absolute value |a _jk -l ₂ | of the difference between all the saddle point distances a _jk and the saddle point distance l ₂ of the two feature pieces (6) on the stick (2), if |a _j2,k2 -l ₂ | value is the smallest, then it is determined that the corresponding two saddle points j2 and k2 are the center points of the two feature pieces (6) on the boom; the inclination angle of the line connecting the two center points is α ₂ : Calculate the absolute value |a _jk -l ₃ | of the difference between all the saddle point distances a _jk and the saddle point distance l ₃ of the two feature pieces (6) on the bucket (3), if |a _j3,k3 -l ₃ | The value is the smallest, then it is determined that the corresponding two saddle points j3 and k3 are the center points of the two feature pieces (6) on the boom; the inclination angle of the line connecting the two center points is α ₃ : F. Narrowing down the detection area: Find the leftmost center point coordinate L(x _l , y _l , 0) and the rightmost center point in the world coordinate system coordinates of the center points of the six currently detected feature sheets (6). Point coordinate R(x _r ,y _r ,0), the uppermost central point coordinate T(x _t ,y _t ,0) and the lowermost central point coordinate B(x _b ,y _b ,0), x ₀ is set The maximum distance of lateral movement of all feature sheets (6) within the detection time interval, and _y0 is the maximum distance of longitudinal movement of all feature sheets within the detection time interval; It is: LB(x _l -x ₀ ,y _l -y ₀ ,0), TR(x _r +x ₀ ,y _r +y ₀ ,0), and then through the inverse transformation of the coordinates, the next rectangular detection area can be obtained The image coordinate system coordinates of the two diagonal points, and then determine the rectangular detection area in the image coordinate system for the next detection; G. Repeat steps B-F to dynamically measure the posture of the excavator working device in real time.