CN117211768A - State detection method and control unit for drilling machine - Google Patents

Abstract

Disclosed is a state detection scheme for a drilling machine, comprising: an image acquisition device (10) arranged on a loading vehicle (2) of the drilling machine is utilized to acquire images at least comprising a hole (9) drilled by the drilling machine and a drill bit (8) of the drilling machine; identifying at least a hole (9) and a drill bit (8) from the image, and determining location information of key points on the hole (9) and the drill bit (8); determining the position of the hole (9) by using the position information of the key point on the hole (9), and determining the pose of the drill bit (8) by using the position information of the key point on the drill bit (8); and expressing the position information of the hole (9) and the pose information of the drill bit (8) in an expression coordinate system.

Description

State detection method and control unit for drilling machine

Technical Field

The present application relates to a state detection (monitoring) scheme for a drilling machine capable of providing at least pose information about a drill bit.

Background

A drilling machine is a construction machine that performs a drilling operation. The drilling operation is generally in a severe environment, and the machine hand often needs to bear high noise, strong vibration, strong sunlight and the like. In order to drill a hole, the drill is usually operated in a fixed working position for a long time, and the operator often repeatedly performs a cyclic action for a long time, such as drilling, boring, lifting, throwing, etc. In order to ensure drilling accuracy, it is necessary to accurately detect and control the pose of the drill bit in each action. In addition, in order to realize the automation of the repeatedly executed cyclic actions, the pose of the mast, the pose of the power head and the position of the unlocking pressure lever need to be automatically detected and determined.

In the prior art, the drill bit height is determined mainly by using information of a rotation speed sensor related to a main hoisting motor of a drilling machine. The method of determining the height of the drill bit in this indirect manner has some drawbacks. First, the zero position needs to be calibrated frequently. Second, there is a cumulative error.

On the other hand, one solution in the prior art regarding the position of the drill bit central axis relative to the hole central axis is to use information of the rotary encoder related to the on-board turning. Since only the on-coming rotational position can be detected based on the rotary encoder information, the drill bit is also moved in other directions in a plane perpendicular to the central axis of the drill bit. In this way, the rotary encoder position (i.e., null) must be determined when the bit center axis coincides with the hole center axis. If the drill is not operationally moved during operation, the position of the drill bit center axis relative to the hole center axis is not known. If the position of the drill bit center axis relative to the hole center axis is determined using information from a rotational speed sensor associated with the on-board swing motor, then accumulated errors may occur in addition to the above-described problems.

Disclosure of Invention

It is an object of the present application to provide a condition detection scheme for a drilling machine that solves at least some of the aforementioned problems of the prior art.

To this end, the present application provides in one of its aspects a condition detection (or monitoring) method for a drilling machine, comprising:

acquiring images at least comprising holes drilled by the drilling machine and drill bits of the drilling machine by using an image acquisition device arranged on an upper vehicle of the drilling machine;

identifying at least a hole and a drill bit from the image, and determining location information of key points on the hole and the drill bit;

determining the position of the hole by using the position information of the key points on the hole, and determining the pose of the drill by using the position information of the key points on the drill; and

and expressing the position information of the hole and the pose information of the drill bit in an expression coordinate system.

In one embodiment, the expression coordinate system is:

the hole alignment coordinate system has its origin of coordinates set on the drill, preferably on the center axis of rotation of the upper carriage, e.g. in a plane bordering between the upper carriage and the lower carriage of the drill and perpendicular to the center axis of rotation of the upper carriage, one axis of the hole alignment coordinate system perpendicularly intersecting the hole center axis of the hole.

In one embodiment, the expression coordinate system is:

the origin of the whole car coordinate system is arranged on the drilling machine, preferably on the rotation center shaft of the upper car, for example, in a plane which is positioned at the junction between the upper car and the lower car of the drilling machine and is perpendicular to the rotation center shaft of the upper car, and the origin of the whole car coordinate system can be coincident with the origin of coordinates of the opposite hole coordinate system, and one shaft of the whole car coordinate system is along the rotation center shaft of the upper car.

In one embodiment, position information of the hole is determined by using on-board coordinates of key points on the hole in the on-board coordinate system, and pose information of the drill is determined by using on-board coordinates of key points on the drill in the on-board coordinate system; and is also provided with

Converting the position information of the hole in the upper vehicle coordinate system and the pose information of the drill bit into the position information of the hole in the expression coordinate system and the pose information of the drill bit through the conversion relation between the upper vehicle coordinate system and the expression coordinate system;

the origin of the upper car coordinate system is set on the drilling machine, preferably on the rotation center shaft of the upper car, for example, in a plane which is located at the junction between the upper car and the lower car of the drilling machine and is perpendicular to the rotation center shaft of the upper car, the origin of the upper car coordinate system can be coincident with the origin of the opposite hole coordinate system and/or the origin of the whole car coordinate system, and one shaft of the upper car coordinate system is along the rotation center shaft of the upper car.

In one embodiment, the conversion relationship between the on-vehicle coordinate system and the opposite-hole coordinate system is determined based on the following information:

get-on coordinates in a get-on coordinate system based on the hole or the orifice center point at the upper end of the inner casing; and/or

Detecting information based on a rotating speed sensor of the on-board rotary motor; and/or

Based on detection information of the angle encoder of the boarding car.

In one embodiment, the conversion relationship between the get-on coordinate system and the hole-alignment coordinate system is determined by:

determining a mapping point of a reference point on a center line of a hole penetrated by a drill rod in a power head in a reference plane, wherein the reference plane is perpendicular to a central axis of a drill bit, and a hole center point of the upper end of the hole or an inner casing thereof is positioned in the reference plane;

and determining a conversion relation between the on-vehicle coordinate system and the opposite-hole coordinate system based on the position difference of the mapping point and the orifice center point.

In one embodiment, the key points on the hole are at least three points on the upper edge of the hole or its inner casing, and the hole position information is the position of the center point of the hole or the orifice at the upper end of its inner casing.

In one embodiment, the pose information of the drill bit includes at least: the position of one, preferably at least two, central points on the drill bit, and the orientation of the central axis of the drill bit.

In one embodiment, the image acquired by the image acquisition device further includes a drill bit association component, and the state detection method further includes:

Identifying a bit-related component from the image and determining location information of a key point on the bit-related component;

determining pose information of the drill bit associated component by utilizing position information of key points on the drill bit associated component; and

expressing pose information of the bit associated part in an expression coordinate system;

wherein the bit-associated components include one or more of:

a power head;

a mast, in particular a lower part of the mast;

and (3) drilling rod.

In one embodiment, the state detection method further includes verifying or correcting the position information of the drill bit based on detection information of a rotational speed sensor of the main winding motor of the drilling machine.

In one embodiment, the state detection method further comprises processing the image to obtain an ambient dense map;

the environment dense map is expressed in the expression coordinate system.

In one embodiment, the state detection method further comprises determining a pose of an image acquisition device based on the image;

and expressing the pose of the image acquisition device in the expression coordinate system.

In one embodiment, the image acquisition device comprises one or more of the following:

a monocular camera;

Binocular cameras;

RGB-D camera;

a laser radar;

millisecond wave radar.

The application also provides a control unit for a drilling machine configured to perform the state detection method of the application.

The application also provides a drilling machine (in particular a rotary drilling machine), comprising:

the image acquisition device is arranged on the upper vehicle of the drilling machine and is configured to acquire images at least comprising holes drilled by the drilling machine and drill bits of the drilling machine; and

the control unit of the application is configured to execute the state detection method of the application based on the information acquired by the image acquisition device.

The present application also provides a machine-readable storage medium storing executable instructions that when executed by a processor implement the state detection method of the present application.

According to the state detection scheme provided by the application, the pose information of the drill bit of the drilling machine can be at least determined based on the information acquired by the image acquisition device. The determined bit pose has high precision, can eliminate accumulated errors and improves drilling precision. The bit pose information determined based on the information directly sensed by the image acquisition device can be used in other auxiliary control schemes of the drilling machine, so that the drilling precision and efficiency can be improved, and the workload of a machine operator can be reduced.

Drawings

The foregoing and other aspects of the application will be more fully understood and appreciated from the following detailed description taken with reference to the accompanying drawings, in which:

FIG. 1 is a schematic view of a drilling rig according to the present application;

FIGS. 2-4 illustrate three coordinate systems used in one detection scheme of the present application;

FIGS. 5-7 are schematic illustrations of key points on a drill work member detected in one detection scheme of the present application;

FIG. 8 is a schematic diagram of key points on a well being tested in one test protocol of the present application;

FIG. 9 is a schematic illustration of rig environment information detected in one detection scheme of the present application;

fig. 10 is a schematic diagram of a drill bit path detected in one detection scheme of the present application.

Detailed Description

The present application relates generally to a condition detection (or monitoring) scheme for a drilling machine, particularly a rotary drilling machine. In a basic embodiment, at least the pose of the drill bit of the drilling machine is detected (monitored). In some further embodiments, environmental information of the drill is detected in addition to the pose of the drill bit.

The state detection scheme related to the application is applicable to various drilling rigs. For convenience of description, the present application is shown in a high degree of outline in fig. 1 as a drilling machine to which the present application is applied. The drilling machine mainly comprises: a lower carriage 1 comprising running means (such as tracks or the like) to enable the whole drill to run; an upper carriage 2 rotatably mounted on the lower carriage 1 about a rotation center axis; a mast 3 pivotally supported on the upper carriage 2 by a carrying mechanism (typically a hydraulic carrying mechanism) 4, typically in a vertical position during a drilling operation; a drill rod 5 which is pulled by a steel wire rope 6 of the main hoisting system and can vertically ascend and descend; a power head 7, one end of which is supported by the mast 3 and can vertically move along the mast 3, and the other end of which can drive the drill rod 5 to rotate; a drill bit (drill pipe) 8 mounted at the lower end of the drill pipe 5 and following the drill pipe 5. The drill bit 8 is used to drill a hole downwards from a surface, such as the ground, hereinafter referred to as a drilling surface, for example a hole 9 of a length schematically shown in fig. 1. The hole 9 has a central axis (hereinafter referred to as hole central axis) 90. The position and orientation of the hole center axis 90 is predetermined, typically perpendicular to the drilling surface, or inclined at a predetermined angle relative to the drilling surface in a certain direction. The position of the upper end of the hole 9 is characterized by a center point (hereinafter referred to as the orifice center point) 95. It is noted that if a casing is provided in the hole 9 and the upper edge of the casing is higher than or equal to the drilling surface, the orifice center point 95 here refers to the center point of the upper edge of the casing. If no casing is provided, or if the upper edge of the casing is below the drilling surface, the orifice center point 95 is referred to herein as the center point of the hole 9.

The drill bit 8 has a central axis (hereinafter referred to as drill bit central axis) 80, which coincides with the central axis of the drill rod 5. The drill bit 8 can rotate with the drill rod 5 when the power head 7 drives the drill rod 5 to rotate, and can be lifted and lowered with the drill rod 5 when the wire rope 6 is paid out or reeled in to lift or lower the drill rod 5.

In drilling holes having a certain depth, the drilling machine cannot be completed through one action, but is generally completed through a repetitive cyclic action. Each cycle typically includes tripping, drilling, lifting, and throwing the earth. And a casing (not shown, e.g., a follow-up casing) may also be disposed in the hole that has been drilled to support the earth surrounding the hole.

After a set of cycling actions is completed (i.e., after the drill bit is thrown out of the earth), it is necessary to reinsert the drill bit 8 into the hole 9 (drill down) as precisely as possible so that the drill bit center axis 80 coincides with the hole center axis 90. For this reason, the pose of the drill bit 8 needs to be accurately determined. The position of the drill bit 8 is typically characterized by the height of the drill bit 8, and the attitude of the drill bit 8 may be characterized by a vector r (directed toward or away from the drill bit 8) coincident with the drill bit central axis 80.

At least for the purpose of accurately determining the pose of the drill bit 8, the present application provides a state detection solution for a drilling machine, which provides for the provision of an image acquisition device 10 on the upper carriage 2, in particular above the upper carriage 2. The image acquisition device 10 may be fixed relative to the upper carriage 2 and thus swivel with the upper carriage 2 relative to the lower carriage 1. And the image capturing apparatus 10 is capable of capturing image information of a facing object. In the present application, the field of view of the image acquisition device 10 needs to be arranged to cover at least the working parts of the rig, in particular the drill bit 8 and the power head 7, which are closely related to the position of the hole 9, and also at least the lower part of the mast 3. In addition, the field of view of the image acquisition device 10 also covers the hole 9.

In order to provide the image capturing mechanism 10 with a sufficient field of view, or to minimize distortion of the image captured by the image capturing mechanism 10, two or more image capturing mechanisms (e.g., cameras) may be used, or one image capturing mechanism (e.g., camera) with a large field of view may be used. Alternatively, the image pickup device 10 may be provided on the boarding car 2 in such a manner as to be able to perform a controlled action. The controlled action may be a movement in the transverse and/or longitudinal and/or vertical direction of the drilling machine, and/or a rotation around the transverse and/or vertical direction, etc.

In the state detection scheme of the present application, the image capturing apparatus 10 may be of a type (not limited to): monocular cameras, binocular cameras, RGB-D cameras, millisecond wave radar, laser radar (LiDAR), and the like.

In the state detection scheme of the present application, pose information about the drill bit of the drilling machine, and possibly also drilling machine environment information (especially hole surrounding environment information) is determined using the acquired information of the newly added image acquisition device 10.

In the state detection scheme of the application, besides the information acquired by the newly-added image acquisition device 10 on the drilling machine, the information of the original sensor of the drilling machine can be combined. Information sources that can be employed in the state detection scheme of the present application may include (without limitation):

(1) A monocular camera;

(2) Monocular camera + (rotational speed sensor of main hoist motor and/or rotational speed sensor of upper car swing motor);

(3) Binocular cameras;

(4) Binocular camera + (rotational speed sensor of main hoist motor and/or rotational speed sensor of upper car swing motor);

(5) RGB-D camera;

(6) RGB-D camera+ (rotational speed sensor of main hoist motor and/or rotational speed sensor of upper car rotary motor);

(7) A millisecond wave radar;

(8) Millisecond wave radar + monocular camera;

(9) Millisecond wave radar+ (rotational speed sensor of main hoist motor and/or rotational speed sensor of upper swing motor);

(10) A laser radar;

(11) Laser radar+ (rotational speed sensor of main hoist motor and/or rotational speed sensor of upper car rotary motor);

(12) Monocular camera + millisecond radar + (rotational speed sensor of main hoist motor and/or rotational speed sensor of boarding swing motor);

(13) A combination of different types of image acquisition devices, and so on.

In addition, based on the principles of the present application, those skilled in the art can also devise the use of other combinations of the newly added image capture device 10 and the original sensors of the drilling machine as the information source in the state detection scheme of the present application.

The pose of the drill bit 8 will be expressed in a relatively fixed expression coordinate system arranged relative to the drill. Several coordinate systems provided for implementing the inventive arrangements are discussed below.

First, a whole car coordinate system O-XYZ is selected on the lower car 2. The whole vehicle coordinate system is a Cartesian coordinate system. As shown in fig. 1, the origin of coordinates O is selected on the center axis of rotation of the upper carriage 2, for example, in a plane (hereinafter referred to as a boundary plane) which is perpendicular to the center axis of rotation of the upper carriage 2 and which is a boundary between the upper carriage 2 and the lower carriage 1, that is, an intersection point of the center axis of rotation of the upper carriage 2 and the boundary plane is the origin O of the entire vehicle coordinate system. Of course, the origin of the entire vehicle coordinate system may be selected at other positions on the center axis of rotation of the upper vehicle 2.

The X axis of the entire vehicle coordinate system coincides with the center axis of rotation of the upper vehicle 2, and may be directed upward as shown in fig. 1, but may also be directed downward. The Z axis of the whole vehicle coordinate system may be directed directly in front of the lower vehicle 1 as shown in fig. 1 along the longitudinal direction of the drill, but may also be directed directly behind the lower vehicle 1. The Y-axis of the complete vehicle coordinate system is not shown in fig. 1, but it will be appreciated that the Y-axis may follow the right hand rule of the cartesian coordinate system along the rig transverse direction. Of course, the axes of the whole vehicle coordinate system can be selected as other directions, and the origin can be selected as other positions.

Next, referring to fig. 2, a coordinate system of the boarding vehicle 2, hereinafter referred to as a boarding coordinate system, is established. The on-vehicle coordinate system is a Cartesian coordinate system, the origin of coordinates is the same as the origin O of the whole vehicle coordinate system, and the on-vehicle coordinate system is represented by O-X _u Y _u Z _u And (3) representing. X is X _u The axis coincides with the X axis of the whole vehicle coordinate system, Z _u The shaft may be directed directly in front of the upper carriage 2 as shown in fig. 2 in the longitudinal direction of the upper carriage 2, but may also be directed directly behind the upper carriage 2. Y of the boarding coordinate system _u The axes are not shown in FIG. 2, but it is understood that Y _u The axis may follow the right hand rule of a cartesian coordinate system in the transverse direction of the upper carriage 2. The upper vehicle coordinate system has only rotational movement about the rotational center axis relative to the entire vehicle coordinate system. Enabling the loading coordinate system to be relative to the whole vehicleRotation angle alpha of mark system around upper vehicle rotation center shaft _u . Of course, the axes of the on-vehicle coordinate system may be selected to be in other directions and the origin may be selected to be in other positions.

Next, referring to fig. 3, a counter-hole coordinate system is established, the counter-hole coordinate system is a cartesian coordinate system, the origin of coordinates is the same as the origin O of the whole vehicle coordinate system, and the counter-hole coordinate system is represented by O-X _h Y _h Z _h And (3) representing. X is X _h The axis coincides with the X axis of the whole vehicle coordinate system, Z _h The axis may be oriented from the origin O toward the hole center axis 90 at a perpendicular line from the origin O toward the hole center axis 90, or from the hole center axis 90 toward the origin O. Y of the opposite hole coordinate system _h The axis is not shown in FIG. 3, but may be defined by X _h Axis and Z _h The axis determination, direction may follow the right hand rule of a Cartesian coordinate system. Of course, other directions may be selected for each axis of the hole coordinate system and other positions may be selected for the origin.

The upper carriage coordinate system has only rotational movement about the rotational center axis relative to the opposite bore coordinate system. The rotation angle of the upper car coordinate system around the upper car rotation center shaft relative to the opposite hole coordinate system is alpha _h 。

Only the rotation motion around the on-vehicle rotation center shaft exists between the opposite hole coordinate system and the whole vehicle coordinate system, and the rotation angle of the opposite hole coordinate system relative to the whole vehicle coordinate system around the on-vehicle rotation center shaft can pass through the rotation angle alpha _u And alpha _h And (5) calculating.

Next, assuming that the image pickup device 10 is a monocular camera, as shown in fig. 4, a camera coordinate system is established. Camera coordinate system with O _c -X _c Y _c Z _c And (3) representing. Origin O of camera coordinate system _c Is the optical center of the camera, Z _c The axis is along the optical axis of the camera, X _c Axes and Y _c The axes are parallel to the u-axis and v-axis, respectively, of the camera's image pixel coordinate system (not shown).

It should be noted that if other forms of image acquisition apparatus 10 are employed, the camera coordinate system referred to in the present application may be replaced by a corresponding image acquisition apparatus coordinate system.

According to conventional algorithms in the field of computer image processing, a conversion relationship between an image pixel coordinate system of a camera and an on-vehicle coordinate system (typically expressed in a combination of an internal matrix and an external matrix) is determined, wherein the internal matrix represents the conversion relationship between the image pixel coordinate system of the camera and the camera coordinate system, and the external matrix represents the conversion relationship between the camera coordinate system and the on-vehicle coordinate system. In addition, the distortion correction of the camera image may also be implemented by conventional algorithms in the field of computer image processing (such as using a distortion matrix, etc.).

In addition, according to a conventional algorithm in the field of space kinematics or space mechanics, the conversion relationship between the coordinate systems, such as the conversion relationship between the camera coordinate system and the upper vehicle coordinate system, the conversion relationship between the upper vehicle coordinate system and the entire vehicle coordinate system, the conversion relationship between the opposite hole coordinate system and the entire vehicle coordinate system, and the conversion relationship between the opposite hole coordinate system and the upper vehicle coordinate system, has a clear expression form, and is usually in the form of a conversion matrix. At this point, it should be noted that whether the camera is fixedly mounted or movably mounted on the boarding car 2, since the pose of the camera with respect to the boarding car 2 is determined at the time of imaging, the conversion relationship between the camera coordinate system and the boarding car coordinate system is determined. For other forms of image acquisition devices, the conversion relationship between the image acquisition device coordinate system and the on-board coordinate system is also determined.

In order to process the image shot by the camera, some key points are set on the working parts which the camera needs to acquire and the hole 9 (or the protection barrel). These key points will be used in image processing. Key points on the working part may be determined by deep learning and should be chosen as points that are easily identifiable and that are capable of characterizing the part geometry, typically points on the part surface.

With respect to the mast 3, taking into account the field of view of the camera or other image acquisition device, it may only be necessary to acquire images of the lower portion thereof. For this purpose, as schematically shown in fig. 5, a projection is chosen as an exemplary mast key point 31, 32, 33, 34 in the lower part of the mast 3. Of course, other apparent points on the lower part of the mast 3 can also be chosen as mast key points.

For the power head 7, as schematically illustrated in fig. 6, the power head keypoints selected on the power head 7 may comprise keypoints 71, 73 near the drill rod 5 side, keypoints 72, 74 near the mast 3 side, two diametrically opposed keypoints 75, 76 on the outer circumference of the drive section for driving the drill rod 5. The centre point 77 of the drive part (lying on the centre axis of the drill rod 5) is located between the key points 75, 76. Of course, other apparent points on the power head 7 may be selected as the power head key points. Furthermore, a point on the center line of the hole in the power head 7 through which the drill rod 5 passes is selected as a reference point, for example, a reference point 78 shown in fig. 6. The reference point 78 is located on the central axis of the drill pipe 5, selected at any suitable location, and the geometric positional relationship between the reference point 78 and other key points on the power head 7 is known. The foot of the reference point 78 in a reference plane containing the orifice center point 95 and perpendicular to the drill bit center axis 80 is the map point 88 (shown in fig. 1), i.e., the map point of the drill bit center axis 80 in the reference plane. The reference plane may or may not coincide with the drilling plane.

For drill bit 8, as schematically illustrated in FIG. 7, the drill bit key points selected on drill bit 8 may include two diametrically opposed key points 81, 82 on the outer periphery of the upper edge of the body of drill bit 8, and two diametrically opposed key points 83, 84 on the outer periphery of the lower edge of the body of drill bit 8. The upper center point 85 of the bit 8 body is located between the keypoints 81, 82 and the lower center point 86 of the bit 8 body is located between the keypoints 83, 84. Both center points 85, 86 lie on the central axis 80 of the drill bit 8 (coinciding with the central axis of the drill rod 5). The feature 87 on the actuating element (plunger) for opening and closing the bottom plate of the drill bit 8 may also be selected for determining the opening and closing of the bottom plate of the drill bit 8. Of course, other apparent points on the drill bit 8 may be selected as drill bit key points.

For the holes 9, as schematically shown in fig. 8, four evenly distributed key points 91, 92, 93, 94 on their upper edges can be chosen. An orifice center point 95 is located between keypoints 91 and 92 and also between keypoints 93 and 94. It will be appreciated that if a casing is provided in the hole 9 and the upper edge of the casing is at or above the drilling surface, the keypoints 91 and 92, 93 and 94 are referred to herein as keypoints on the upper edge of the casing. If no casing is provided, or if the upper edge of the casing is below the drilling surface, the keypoints 91 and 92, 93 and 94 are referred to herein as keypoints of the hole 9.

An exemplary state detection method based on the monocular camera of the present application is described next.

Firstly, the camera captures a picture in the field of view, wherein the picture at least comprises the power head 7, the drill bit 8, the hole 9 and the lower part of the mast 3.

Next, distortion correction is performed on the picture. This can be achieved by existing methods, such as Zhang Zhengyou calibration methods, and the like.

Next, by means of existing image recognition (e.g. image segmentation, in particular instance segmentation) and keypoint detection techniques, a pattern of the mast 3, the power head 7, the drill bit 8, the bore 9 (or the casing) is obtained from the corrected picture (e.g. using masking techniques), and the keypoints in the pattern and the pixel coordinates of these keypoints in the image pixel coordinate system are determined.

Next, the pixel coordinates of the center point 77 are calculated from the pixel coordinates of the key points 75 and 76, the pixel coordinates of the center point 85 are calculated from the pixel coordinates of the key points 81 and 82, and the pixel coordinates of the center point 86 are calculated from the pixel coordinates of the key points 83 and 84.

Next, the geometric constraint relationship in the boarding coordinate system of these key points and center points, in particular, the boarding coordinates of the points 77, 85, 86, 87, 95, is obtained by using the conversion relationship (internal reference matrix) between the image pixel coordinate system and the camera coordinate system, the conversion relationship (external reference matrix) between the camera coordinate system and the boarding coordinate system, and the key points 31, 32, 33, 34 on the mast 3, the key points 71, 72, 73, 74 on the power head 7, the key points 81, 82, 83, 84 on the drill bit 8, the key points 91, 92, 93, 94 on the hole 9 (or the casing). In addition, since the reference point 78 on the power head 7 has a known geometric relationship with the key points 71, 72, 73, 74, the boarding coordinates of the reference point 78 can be accurately calculated from the boarding coordinates of the key points 71, 72, 73, 74.

As will be appreciated by those skilled in the art, for any of the mast 3, the power head 7, the drill bit 8, the bore 9 (or the casing), 12 equations can be listed, including 12 variables, using the pixel coordinates of the four key points and the geometric constraint relationships between them (distance, angle between links, etc.). Solving the 12 equations yields the values of 12 variables, i.e., the on-coming coordinates of the four key points in the on-coming coordinate system (the values of each point on three coordinate axes of the on-coming coordinate system). If more than 4 keypoints are employed, the number of equations listed will be more than the number of variables solved. In this case, the boarding coordinates of each key point can be determined by using an optimization algorithm such as a least square method, so that the determined boarding coordinates are more accurate.

If there is a specific true geometric relationship between key points in a certain graph, e.g. certain points in the on-vehicle coordinate system are in a certain direction (e.g. in X _u 、Y _u Or Z is _u In the direction) the same coordinate values, the number of variables to be solved may be reduced. For example, in the case where the number of variables to be solved is reduced to 9, only three pixel coordinates of key points having a specific geometric relationship and geometric constraint relationships (distances, included angles between lines, etc.) therebetween are actually required to list 9 equations, including 9 variables. Solving the 9 equations yields the values of 9 variables, i.e., the on-coming coordinates of the three key points in the on-coming coordinate system (the values of each point on the three coordinate axes of the on-coming coordinate system). In this case, four or even more key points may still be employed, and the number of equations listed will be greater than the number of variables solved. In this case, the boarding coordinates of each key point can be determined by using an optimization algorithm such as a least square method, so that the determined boarding coordinates are more accurate. In addition, regarding the selection of key points, it is desirable that geometric constraints be formed between key points, such as forming specific angles (e.g., right angles) between the lines, and so on.

If the upper edge of the hole 9 (or casing) is flat, then the position of the orifice center point 95 can be calculated by selecting three key points on the upper edge. But using four or more key points on the upper edge of the hole 9 (or casing) may improve the accuracy of the calculated orifice center point 95.

The position of the drill rod 5 (which may be represented by a certain position on the drill rod 5, for example, the position of the connection position of the bottom end of the drill rod 5 and the drill bit 8) and the direction of the drill bit central shaft 80 (represented by a vector r) in the on-vehicle coordinate system are easily obtained by combining the on-vehicle coordinates of the key points on the mast 3, the power head 7 and the drill bit 8 and the geometric constraint relation formed by the mechanical structures between the key points and the drill rod 5. For example, the position (in particular the height) of the drill rod 5 may be determined by the difference between the on-board coordinates of the key point on the drill bit 8 and the on-board coordinates of the key point on the mast 3. The orientation of the drill center axis 80 may be determined by the value of the respective center point 77, 85, 86 or the on-board coordinates of any two of the center points.

Similarly, the on-coming coordinates of the center point 77 of the drive section of the power head 7, the center points 85, 86 and the feature points 87 on the body of the drill bit 8, the orifice center point 95 in the on-coming coordinate system are also easily calculated, with the center points 77, 85, 86 being located on the drill bit center axis 80.

By means of the boarding coordinates of the aperture centre point 95 and the reference point 78 on the power head 7, and by means of the geometrical relationship between the reference point 78, its mapping point 88 in the reference plane and the aperture centre point 95 (e.g. the line between the mapping point 88 and the reference point 78 points at the same direction as the vector r and the line between the mapping point 88 and the reference point 78 is perpendicular to the line between the mapping point 88 and the aperture centre point 95), the boarding coordinates of the mapping point 88 can be calculated.

From the coordinates of the orifice center point 95 in the boarding coordinate system and the coordinates of the orifice center point 95 in the opposite-hole coordinate system, the rotation angle α of the boarding coordinate system about the boarding rotation center axis with respect to the opposite-hole coordinate system can be determined _h The conversion relation, typically a conversion matrix, between the on-board coordinate system and the opposite hole coordinate system is thus determined.

By utilizing the conversion relation between the on-vehicle coordinate system and the opposite hole coordinate system, the coordinate values of the key points, the center points, the mapping points 88, the vector r and the like in the opposite hole coordinate system, in particular to the opposite hole coordinates of the points 77, 85, 86, 87, 95 and 88 can be calculated, namely pose information of each relevant working part, in particular the drill bit 8, in the opposite hole coordinate system is determined. The pose information of the drill bit 8 may be characterized by the orientation of the drill bit central axis 80 and the coordinate values of certain important points on the drill bit 8 (e.g., one or both of the center points 85, 86, point 87, etc.). The position of the hole 9 is characterized by the coordinate value of the orifice center point 95.

The pose information of the working parts, in particular the drill bit 8, in the opposite hole coordinate system contributes to the precise positioning of the drill bit 8 with respect to the hole 9 and to the realization of certain auxiliary functions, in particular automatic functions, of the drilling machine. In addition, the positional information of the feature points 87 (representing the actuating elements for the floor) on the drill bit 8 (whether in the opposite hole coordinate system or the on-board coordinate system) may be used in the control logic that automatically controls the opening of the floor of the drill bit 8 and to assist in determining whether the floor of the drill bit 8 is open.

Because the conversion relation between the camera coordinate system and the upper vehicle coordinate system is determined, the conversion relation between the camera coordinate system and the opposite hole coordinate system can be determined by combining the conversion relation between the upper vehicle coordinate system and the opposite hole coordinate system determined as described above, so that the pose of the camera in the opposite hole coordinate system can be determined.

It can be understood that, based on the basic principles of space kinematics and space mechanics, in the case that the coordinates of some points on an object are known, solving the coordinates of other points or the direction of a certain straight line is merely a computational problem; in addition, the conversion of the coordinates of a certain point or the pointing direction of a certain line between different coordinate systems is also merely a computational problem. And will therefore not be discussed in detail herein.

Since the conversion calculation between the coordinate systems is easy to realize, although the description has been made above of the respective working members, the holes 9 (or the casings) being expressed in the opposite hole coordinate system, it is easy to express these information in the whole vehicle coordinate system, even the upper vehicle coordinate system, by the conversion between the coordinate systems. Alternatively, other suitable coordinate systems may be selected as the expression coordinate system to express the position information of the component of interest.

In general, in the state detection scheme of the present application, pose information of a working member to be focused in an expression coordinate system can be determined based on signals acquired by an image acquisition device. In particular, the pose of the drill bit 8 (including the position of the feature point 87 on the actuating element of the base plate), the position of the hole 9 (or casing) and the pose information of the mast 3 can be determined based on the mast keypoints 31, 32, 33, 34 or other keypoints and the pose information of the power head 7 can be determined based on the keypoints 71, 72, 73, 74 or other keypoints on the power head 7 as described above. Such information may be used for various rig functions including drilling accuracy control, automatic control functions, and the like.

In the state detection scheme based on the monocular camera, further, based on the distortion corrected picture, the pose of the camera in the boarding coordinate system can be acquired and a dense map of the drilling machine environment, particularly an environment dense map around the hole 9, can be constructed by utilizing the existing computer vision algorithm, such as a Slam algorithm. By means of the conversion relation between the camera coordinate system and the on-vehicle coordinate system, and the on-vehicle coordinate values of the key points and the center points of the hole 9, the pose of the camera in the on-vehicle coordinate system (which can be used for verifying or correcting the pose of the camera determined in the on-vehicle coordinate system) can be obtained, and a dense map of the environment around the hole 9 in the on-vehicle coordinate system can be constructed, as schematically shown in fig. 9. In fig. 9, a pile 11 of earth formed by throwing earth outside the hole 9 of the drill bit 8 is schematically represented in the on-board coordinate system. Points 12, 13, 14 marked on the upper contour of the pile 11 correspond to the respective soil throwing positions of the drill bit 8. In fig. 10, the path (indicated by a segment of a circular arc) that the drill bit 8 passes from the hole 9 for the throwing of earth is depicted in the reference plane, on which path each throwing point 12, 13, 14. With the aid of fig. 9 and/or 10, the next soil dump position can be planned.

It should be noted that, regarding the rotation angle α of the upper carriage coordinate system with respect to the opposite hole coordinate system about the upper carriage rotation center axis _h In addition to the calculation using the boarding coordinates of the orifice center point 95 described aboveIn addition to the above, the rotation speed sensor of the boarding swing motor may be used. The speed sensor of the upper swing motor marks a zero position at the orifice center point 95. The upper vehicle 2 rotates by a rotation angle alpha _h After that, the rotation angle alpha _h The speed of the motor can be obtained through the integral of the speed measured by a speed sensor of the on-board rotary motor, or can be calculated through the accumulated number of teeth in the on-board rotary motor. Alternatively, the angle alpha _h Or can be measured directly by an angle encoder provided for the boarding car 2. Corner alpha _h The value calculated previously by the boarding coordinates of the orifice center point 95 may be used, or the value obtained by the rotational speed sensor of the boarding swing motor may be used, or the value directly measured by an angle encoder. Corner alpha _h The individual values may be mutually verified to improve accuracy and reliability.

It should also be pointed out that in the monocular camera based condition detection scheme, if the signals of the rotational speed sensor of the main hoisting motor are employed, it may be used to verify or correct the calculated positions of the drill rod 5 and the drill bit 8.

It should also be noted that the upper vehicle coordinate system is rotated about the rotation angle alpha of the upper vehicle rotation center axis relative to the entire vehicle coordinate system, if necessary _u And can also be determined. For example, the rotation angle α can be calculated by using the on-vehicle coordinates of the key points of some parts on the off-vehicle 1 acquired by the image acquisition device 10 in the on-vehicle coordinate system _u . Alternatively, the rotation angle α may be obtained by a rotation speed sensor of the boarding swing motor or an angle encoder provided for the boarding vehicle 2 _u . If a rotation speed sensor is to be used to obtain alpha _u Because the artificially set zero point of the rotation speed sensor is the zero point of the hole coordinate system, the zero point of the hole coordinate system is not necessarily the zero point of the whole vehicle coordinate system, and an angle alpha which needs to be measured in advance can be formed between the two zero points _offset Then alpha is _u ＝α _offset +α _h 。

At the angle alpha _u After the determination, the conversion relation (usually conversion matrix) of the upper vehicle coordinate system relative to the whole vehicle coordinate system can be determined, and then the mast 3, the power head 7, the drill bit 8, the hole 9 (or the pile casing) and the relevant points can be positioned at the following positionsThe coordinate values in the on-vehicle coordinate system are converted into coordinate values in the whole vehicle coordinate system, namely the pose of the drill bit 8 and the hole 9 (or the casing) can be constructed in the whole vehicle coordinate system. Likewise, dense maps of rig environment can also be constructed in the complete vehicle coordinate system.

It is also noted that the coordinates of the respective points described above in the respective coordinate systems are determined. However, some of these points may be omitted for different rig functions and there may be other points that need to be added.

An exemplary status detection (monitoring) method based on a binocular camera or an RGB-D camera is described next.

First, a camera (binocular camera or RGB-D camera) captures pictures in its field of view.

Next, distortion correction is performed on the picture.

Next, by means of existing image recognition (e.g. image segmentation, in particular instance segmentation) and keypoint detection techniques, the figures of the mast 3, the drill pipe 5, the power head 7, the drill bit 8, the hole 9 or the casing, the ground are obtained from the corrected pictures (e.g. by means of masking techniques), and the keypoints in the figures and the pixel coordinates of these keypoints in the image pixel coordinate system are determined.

Next, using the pixel coordinates of some key points and the depth of field of the camera, the on-board coordinates of the mast 3, the drill pipe 5, the power head 7, the drill bit 8, the hole 9 or the casing, the ground and the key points on the above are obtained through the existing 3D reconstruction technology.

Next, the boarding coordinates of the orifice center point 95 are estimated from the boarding coordinates of the key points on the hole 9 or casing.

Next, the direction and positioning point of the drill rod center axis in the loading coordinate system are estimated by one of the following two ways:

(a) Utilizing the loading coordinates of the edge profile of the drill rod 5;

(b) The mechanical (geometrical) constraint relation between the key point and the central shaft of the drill rod is utilized by the on-car coordinates of the key point on the power head 8.

Next, the boarding coordinates of the respective center points 77, 85, 86 are obtained by one of the following two ways:

(a) Using the boarding coordinates of the keypoints 75, 76, 81, 82, 83, 84;

(b) The upper coordinates of a third point, taken on the respective circles of the three groups of points (75 and 76), (81 and 82), (83 and 84), and the mechanical (geometrical) constraint relationship between the third point and the central axis of the drill rod are used.

Next, the on-carriage coordinates of the mapping points 88 of the drill bit center axis 80 in the reference plane are calculated.

From the coordinates of the orifice center point 95 in the boarding coordinate system and the coordinates of the orifice center point 95 in the opposite-hole coordinate system, the rotation angle α of the boarding coordinate system about the boarding rotation center axis with respect to the opposite-hole coordinate system can be determined _h The conversion relation, typically a conversion matrix, between the on-board coordinate system and the opposite hole coordinate system is thus determined.

By utilizing the conversion relation between the on-vehicle coordinate system and the opposite hole coordinate system, coordinate values of each key point, the center point, the mapping point 88, the vector r and the like in the opposite hole coordinate system, in particular to opposite hole coordinates of the points 77, 85, 86, 87, 95 and 88, namely pose information of each relevant working part, in particular the drill bit 8, in the opposite hole coordinate system can be calculated.

Further, based on the distortion corrected picture, the pose of the camera in the on-vehicle coordinate system can be obtained and a dense map of the drilling machine environment can be constructed by using the existing computer vision algorithm, such as a Slam algorithm, and the pose of the camera in the on-vehicle coordinate system can be obtained and the dense map of the environment around the hole 9 in the on-vehicle coordinate system can be constructed by means of the conversion relation between the on-vehicle coordinate system and the on-vehicle coordinate system.

The description of the same or similar features in the binocular camera or RGB-D camera based state detection scheme as in the previous exemplary flow for the monocular camera based state detection scheme is omitted.

The state detection scheme based on the millisecond wave radar is similar to that based on the monocular camera, and a description thereof will not be repeated here.

An exemplary state detection (monitoring) method based on the monocular camera combined millisecond wave radar is described next.

Firstly, a camera acquires pictures in the field of view of the camera, and a millisecond wave radar acquires data.

Next, distortion correction is performed on the picture.

Next, by means of existing image recognition (e.g. image segmentation, in particular instance segmentation) and keypoint detection techniques, the figures of the mast 3, the drill pipe 5, the power head 7, the drill bit 8, the hole 9 or the casing, the ground are obtained from the corrected pictures (e.g. by means of masking techniques), and the keypoints in the figures and the pixel coordinates of these keypoints in the image pixel coordinate system are determined.

Next, using the pixel coordinates of some key points and depth of field obtained by the millisecond wave radar, the mast 3, the drill rod 5, the power head 7, the drill bit 8, the hole 9 or the casing, the ground and the boarding coordinates of each key point on the mast are obtained through the existing 3D reconstruction technology.

Next, the boarding coordinates of the orifice center point 95 are estimated from the boarding coordinates of the key points on the hole 9 or casing.

Next, the direction and positioning point of the drill rod center axis in the loading coordinate system are estimated by one of the following two ways:

(a) Utilizing the loading coordinates of the edge profile of the drill rod 5;

(b) The mechanical (geometrical) constraint relation between the key point and the central shaft of the drill rod is utilized by the on-car coordinates of the key point on the power head 8.

Next, the boarding coordinates of the respective center points 77, 85, 86 are obtained by one of the following two ways:

(a) Using the boarding coordinates of the keypoints 75, 76, 81, 82, 83, 84;

(b) The upper coordinates of a third point, taken on the respective circles of the three groups of points (75 and 76), (81 and 82), (83 and 84), and the mechanical (geometrical) constraint relationship between the third point and the central axis of the drill rod are used.

Next, the on-carriage coordinates of the mapping points 88 of the drill bit center axis 80 in the reference plane are calculated.

From the coordinates of the orifice center point 95 in the boarding coordinate system and the coordinates of the orifice center point 95 in the opposite-hole coordinate system, the rotation angle α of the boarding coordinate system about the boarding rotation center axis with respect to the opposite-hole coordinate system can be determined _h The conversion relation, typically a conversion matrix, between the on-board coordinate system and the opposite hole coordinate system is thus determined.

By utilizing the conversion relation between the on-vehicle coordinate system and the opposite hole coordinate system, coordinate values of each key point, the center point, the mapping point 88, the vector r and the like in the opposite hole coordinate system, in particular to opposite hole coordinates of the points 77, 85, 86, 87, 95 and 88, namely pose information of each relevant working part, in particular the drill bit 8, in the opposite hole coordinate system can be calculated.

Further, based on the distortion corrected picture, the pose of the camera in the on-vehicle coordinate system can be obtained and a dense map of the drilling machine environment can be constructed by using the existing computer vision algorithm, such as a binocular Slam algorithm, and the pose of the camera in the on-vehicle coordinate system and the environment dense map around the hole 9 in the on-vehicle coordinate system can be obtained and the on-vehicle coordinate system can be constructed by means of the conversion relation between the on-vehicle coordinate system and the on-vehicle coordinate system.

The description of the same or similar points in the state detection scheme based on the monocular camera combined millisecond wave radar as in the previous exemplary flow for the state detection scheme based on the monocular camera is omitted.

An exemplary condition detection (monitoring) method based on the lidar is described next. It should be noted that the lidar is capable of obtaining a three-dimensional point cloud alone.

First, a three-dimensional point cloud is obtained by a laser radar.

Next, the three-dimensional point cloud is identified (e.g. example segmentation) and the keypoint detected by means of existing techniques, resulting in a mast 3, a drill pipe 5, a power head 7, a drill bit 8, a hole 9 or a casing, and boarding coordinates of each keypoint in a boarding coordinate system.

Next, the boarding coordinates of the orifice center point 95 are estimated from the boarding coordinates of the key points on the hole 9 or casing.

Next, the direction and positioning point of the drill rod center axis in the loading coordinate system are estimated by one of the following two ways:

(a) Utilizing the loading coordinates of the edge profile of the drill rod 5;

(b) The mechanical (geometrical) constraint relation between the key point and the central shaft of the drill rod is utilized by the on-car coordinates of the key point on the power head 8.

Next, the boarding coordinates of the respective center points 77, 85, 86 are obtained by one of the following two ways:

(a) Using the boarding coordinates of the keypoints 75, 76, 81, 82, 83, 84;

(b) The upper coordinates of a third point, taken on the respective circles of the three groups of points (75 and 76), (81 and 82), (83 and 84), and the mechanical (geometrical) constraint relationship between the third point and the central axis of the drill rod are used.

Next, the on-carriage coordinates of the mapping points 88 of the drill bit center axis 80 in the reference plane are calculated.

From the coordinates of the orifice center point 95 in the boarding coordinate system and the coordinates of the orifice center point 95 in the opposite-hole coordinate system, the rotation angle α of the boarding coordinate system about the boarding rotation center axis with respect to the opposite-hole coordinate system can be determined _h The conversion relation, typically a conversion matrix, between the on-board coordinate system and the opposite hole coordinate system is thus determined.

By utilizing the conversion relation between the on-vehicle coordinate system and the opposite hole coordinate system, coordinate values of each key point, the center point, the mapping point 88, the vector r and the like in the opposite hole coordinate system, in particular to opposite hole coordinates of the points 77, 85, 86, 87, 95 and 88, namely pose information of each relevant working part, in particular the drill bit 8, in the opposite hole coordinate system can be calculated.

Further, based on the three-dimensional point cloud, the pose of the camera in the on-vehicle coordinate system can be obtained and a dense map of the drilling machine environment can be constructed by utilizing the existing computer vision algorithm, such as a laser Slam algorithm, and the pose of the camera in the on-vehicle coordinate system can be obtained and the dense map of the environment around the hole 9 in the on-vehicle coordinate system can be constructed by means of the conversion relation between the on-vehicle coordinate system and the on-vehicle coordinate system.

The description of the lidar-based state detection scheme that is the same as or similar to that in the previous exemplary flow for the monocular camera-based state detection scheme is omitted.

It should be noted that although in the examples described above, the posture of the drill bit 8 is determined mainly by using the key points on the drill bit 8 and the power head 7, mast 3 as the representative of the drill bit-related components, the posture of the drill bit 8 may be determined by using the key points on the drill bit 8 and other drill bit-related components (e.g., drill pipe 5).

The method for detecting (or monitoring) the state of the drilling machine according to the application, by means of the image acquisition device and the adaptive algorithm, determines the pose (especially the position of the center point and the central axis direction) of the drilling bit 8 in real time, can be used in various control programs (such as track planning, real-time motion control and the like) of the drilling machine, especially compared with the position of the center point of the hole and the central axis direction of the hole, so that the center point of the drilling bit coincides with the center point of the hole and the central axis direction of the drilling bit coincides with the central axis direction of the hole (or the central axis of the drilling bit coincides with the central axis direction of the hole) as far as possible when the drilling machine is drilled again. Whereby each drilling cycle action can be performed accurately. The pose of the other components of the rig determined in real time by the present application may also be determined by similar methods and used in various control programs of the rig, such as trajectory control and the like.

Those skilled in the art may make various adaptations to the details of the exemplary flow of the state detection method described above, the steps (including the specific contents of the steps and the order of execution), etc. according to specific application scenarios.

In summary, the present application relates to a drilling machine including a control unit (not shown) that receives commands from the command input element and the image acquisition device 10 and feedback signals from the actuators, control valves, and sensors, and controls the operation of the hydraulic system main pump and the actuators, control valves of the drilling machine. The state detection method described above may be provided in the control unit. The control unit performs the state detection process described above based on the signal acquired by the image acquisition device 10.

It is noted that due to Z to the hole coordinate system _h The axes intersect the hole center axis 90, and thus, it may be more intuitive to represent the pose of the drill bit 8, hole 9 (or casing), and other components, a dense map of the environment, in a hole-to-hole coordinate system. However, as previously described, dense maps of the pose, environment, of the drill bit 8, hole 9 (or casing), and other components may also be constructed in the overall vehicle coordinate system, and even the on-vehicle coordinate system, which may be more convenient to use for achieving certain rig functions. Therefore, the opposite hole coordinate system or the whole vehicle coordinate system, or even the on-vehicle coordinate system can be selected as an expression coordinate system for expressing the information according to specific needs.

It should be noted that, although the positions of the drill bit 8 and the hole 9 (casing) are determined by using the coordinates of some key points in the above coordinate system in the previous example, the positions of the drill bit 8 and the hole 9 (casing) may be determined by using the coordinates of some key points in other coordinate systems (for example, the coordinate system of the image capturing device, the coordinate system of the whole vehicle, and the coordinate system of the opposite hole).

The present application also provides a machine-readable (computer-readable) storage medium storing executable instructions that when executed by a processor implement a state detection method as described above.

The application also provides a drilling rig, in particular a rotary drilling rig, comprising the condition detection scheme described above and related structures.

According to the state detection scheme, an image acquisition device capable of directly sensing the information of the drill bit and the hole is added on the drilling machine, and the pose information of the drill bit in an expression coordinate system is determined in real time based on the information acquired by the image acquisition device. The determined bit pose has high precision, and the accumulated error can be eliminated, so that the drilling precision is improved. Further, it is also possible to determine environmental information around the opening, such as pile information, etc., based on the collected information. The bit pose information (and possibly environmental information) determined based on the information directly sensed by the image acquisition device can be used in other auxiliary control schemes (such as automatic track control and the like) of the drilling machine, so that the drilling precision and efficiency can be improved, and the workload of a machine hand can be reduced. Furthermore, according to the state detection scheme of the application, the pose of the components in the expression coordinate system can be determined in real time according to the acquired information of the key points on the mast and the power head, and the pose of the two components is also important for the automatic control of the whole vehicle. Similarly, the positional information of the actuating element (plunger) of the drill floor may also be determined and used in the automatic control of the entire vehicle.

Although the application is described herein with reference to specific embodiments, the scope of the application is not intended to be limited to the details shown. Various modifications may be made to these details without departing from the underlying principles of the application.

Claims (15)

1. A state detection method for a drilling machine, comprising:

an image acquisition device (10) arranged on a loading vehicle (2) of the drilling machine is utilized to acquire images at least comprising a hole (9) drilled by the drilling machine and a drill bit (8) of the drilling machine;

identifying at least a hole (9) and a drill bit (8) from the image, and determining location information of key points on the hole (9) and the drill bit (8);

determining the position of the hole (9) by using the position information of the key point on the hole (9), and determining the pose of the drill bit (8) by using the position information of the key point on the drill bit (8); and

and expressing the position information of the hole (9) and the pose information of the drill bit (8) in an expression coordinate system.

2. The state detection method according to claim 1, wherein the expression coordinate system is:

a hole alignment coordinate system, the origin of which is arranged on the drilling machine, preferably on the rotation center axis of the upper carriage (2), for example in a plane which is located at the junction between the upper carriage (2) and the lower carriage (1) of the drilling machine and is perpendicular to the rotation center axis of the upper carriage (2), one axis of the hole alignment coordinate system is perpendicularly intersected with the hole center axis (90) of the hole (9); or alternatively

The origin of the whole vehicle coordinate system is arranged on the drilling machine, preferably on the rotation center shaft of the upper vehicle (2), for example, in a plane which is located between the upper vehicle (2) and the lower vehicle (1) of the drilling machine and is perpendicular to the rotation center shaft of the upper vehicle (2), the origin of the whole vehicle coordinate system can be coincident with the origin of coordinates of the opposite hole coordinate system, and one shaft of the whole vehicle coordinate system is along the rotation center shaft of the upper vehicle (2).

3. The state detection method according to claim 1 or 2, wherein position information of the hole (9) is determined in a get-on coordinate system by using a get-on coordinate of a key point on the hole (9) in the get-on coordinate system, and pose information of the drill bit (8) is determined by using a get-on coordinate of a key point on the drill bit (8) in the get-on coordinate system; and is also provided with

Converting the position information of the hole (9) in the on-vehicle coordinate system and the pose information of the drill bit (8) into the position information of the hole (9) in the on-vehicle coordinate system and the pose information of the drill bit (8) through the conversion relation between the on-vehicle coordinate system and the expression coordinate system;

the origin of the upper car coordinate system is arranged on the drilling machine, preferably on a rotation center shaft of the upper car (2), for example, in a plane which is located between the upper car (2) and the lower car (1) of the drilling machine and is perpendicular to the rotation center shaft of the upper car (2), the origin of the upper car coordinate system can be coincident with the origin of the opposite hole coordinate system and/or the origin of the whole car coordinate system, and one axis of the upper car coordinate system is along the rotation center shaft of the upper car (2).

4. The state detection method according to claim 3, wherein the conversion relationship between the on-vehicle coordinate system and the counter-hole coordinate system is determined based on the following information:

get-on coordinates in a get-on coordinate system based on an orifice center point (95) of the hole (9) or the upper end of the inner casing thereof; and/or

Detecting information based on a rotating speed sensor of the on-board rotary motor; and/or

Based on detection information of the angle encoder of the boarding car.

5. The state detection method according to claim 3, wherein the conversion relation between the on-vehicle coordinate system and the counter-hole coordinate system is determined by:

-determining a mapping point (88) of a reference point (78) on the centre line of a hole in the power head (7) penetrated by the drill rod (5) in a reference plane, said reference plane being perpendicular to the drill bit centre axis (80), and an orifice centre point (95) of the upper end of the hole (9) or its inner casing being located in said reference plane;

a conversion relationship between the on-board coordinate system and an opposite-hole coordinate system is determined based on a difference in position of the mapping point (88) and the aperture center point (95).

6. The state detection method according to any one of claims 1 to 5, wherein the key points on the hole (9) are at least three points on the upper edge of the hole (9) or the inner casing thereof, and the hole (9) position information is the position of the orifice center point (95) of the upper end of the hole (9) or the inner casing thereof.

7. The state detection method according to any one of claims 1-6, wherein the pose information of the drill bit (8) comprises at least: the position of one, preferably at least two, central points on the drill bit (8) and the orientation of the drill bit central axis (80).

8. The state detection method according to any one of claims 1 to 7, wherein the image acquired by the image acquisition device (10) further includes a bit association part; and the state detection method further includes:

identifying a bit-related component from the image and determining location information of a key point on the bit-related component;

determining pose information of the drill bit associated component by utilizing position information of key points on the drill bit associated component; and

expressing pose information of the drill bit associated component in the expression coordinate system;

wherein the bit-associated components include one or more of:

a power head (7);

mast (3), in particular the lower part of mast (3);

and a drill rod (5).

9. The state detection method according to any one of claims 1 to 8, further comprising verifying or correcting positional information of the drill bit (8) based on detection information of a rotational speed sensor of the main hoisting motor of the drilling machine.

10. The state detection method of any one of claims 1-9, further comprising processing the image to obtain an ambient dense map;

The environment dense map is expressed in the expression coordinate system.

11. The state detection method according to any one of claims 1-10, further comprising determining a pose of an image acquisition device (10) based on the image;

and expressing the pose of the image acquisition device (10) in the expression coordinate system.

12. The state detection method according to any one of claims 1-11, wherein the image acquisition device (10) comprises one or more of the following:

a monocular camera;

binocular cameras;

RGB-D camera;

a laser radar;

millisecond wave radar.

13. A control unit for a drilling machine configured to perform the state detection method of any one of claims 1-12.

14. A drilling rig, comprising:

an image acquisition device (10) arranged on the upper carriage (2) of the drilling machine and configured to acquire images at least comprising a hole (9) drilled by the drilling machine and a drill bit (8) of the drilling machine; and

the control unit of claim 13, configured to perform the state detection method of any one of claims 1-12 based on an image acquired by the image acquisition device (10).

15. A machine-readable storage medium storing executable instructions which when executed by a processor implement the state detection method of any one of claims 1-12.