JP4962415B2 - Cutting edge position measuring device for construction machinery - Google Patents

Description

  The present invention relates to a blade edge position measuring device for measuring a blade edge position of a construction machine.

  The most common method for measuring the position of the cutting edge of a construction machine such as a hydraulic excavator is the visual measurement of an operator who is operating the hydraulic excavator. In addition, a method of measuring a blade edge position by attaching a rotation angle sensor that measures a rotation angle to each joint of a boom, an arm, and a bucket of a hydraulic excavator has been tried (Non-Patent Document 1).

Norio Seomura and 1 other, "Development of excavator instruction system for excavators", 6th Annual Meeting of the Japan Society of Civil Engineers, 6th part, September 2002, p. 921-922 JP 63-004126 A Japanese Patent Laid-Open No. 02-176023 Japanese Patent Laid-Open No. 02-261129

In the hydraulic excavator shown in Non-Patent Document 1 and the conventional hydraulic excavator, the method for measuring the blade edge position has the following problems.
(1) When the blade edge position is measured by the operator's eye measurement, the measurement position varies by the driver. In particular, when the operator is not skilled, it is difficult to measure the correct position of the cutting edge.
(2) When the blade edge position is measured by the operator's eye measurement, it is difficult to digitize the measurement position, and it is difficult to convert the measurement position into data and divert it to other devices.
(3) When measuring the blade edge position with the rotation angle sensor, it is necessary to attach the rotation angle sensor to each joint of the movable parts such as the boom, the arm, and the bucket, and it takes time for the attachment work.
(4) When measuring the blade edge position with the rotation angle sensor, the blade edge position is calculated from the rotation angle of the joint measured by the rotation angle sensor and the length of the movable part such as the boom, arm, and bucket. It needs to be determined accurately. Further, if the hydraulic excavator to be used is changed, the operation for obtaining the length of the movable part needs to be performed for each hydraulic excavator, and the adjustment work becomes complicated.
(5) When the cutting edge position is measured by the rotation angle sensor, the cutting edge position is calculated from the rotation angle of the joint measured by the rotation angle sensor and the length of the movable part such as the boom, arm, and bucket. The measurement position error due to the looseness of the rotary joint portion or the like is accumulated. For this reason, in order to correctly measure the cutting edge position, it is necessary to keep the operating state of the hydraulic excavator itself very well so as to suppress as much as possible play in the rotary joint.
(6) When measuring the blade edge position with the rotation angle sensor, it is necessary to attach the rotation angle sensor to many places of the joints of the movable parts such as the boom, arm, and bucket, so that the failure of the sensor that is a precision instrument occurs. Will increase.

  The present invention has been made in view of the above problems, and an object of the present invention is to provide a cutting edge position measuring device for a construction machine that can accurately measure the cutting edge position of the construction machine.

A cutting edge position measuring device for a construction machine according to a first invention that solves the above-described problems,
A blade position measuring device for a construction machine that measures the blade edge position of a bucket of a construction machine,
A plurality of markers attached to the bucket and arranged on one straight line perpendicular to the rotation axis of the bucket;
At least two imaging means provided at different positions of the construction machine main body for imaging an image including the plurality of markers;
A blade edge position calculator that calculates the blade edge position of the bucket based on the image provided by the construction machine and photographed by the photographing means;
The cutting edge position calculator is
By binarizing the image photographed by the photographing means, the plurality of markers are extracted and positions on the image are detected, and the positions of the plurality of markers are detected by triangulation from the detected positions on the image. Obtain a three-dimensional position, obtain the inclination of a vector connecting the plurality of markers from the obtained three-dimensional position of the plurality of markers, obtain the obtained three-dimensional position of the plurality of markers, obtain the obtained inclination of the vector, and in advance The blade edge position of the bucket is obtained from the offset value of the blade edge position with respect to one of the plurality of markers.

A cutting edge position measuring device for a construction machine according to a second invention that solves the above-described problem is,
A blade position measuring device for a construction machine that measures the blade edge position of a bucket of a construction machine,
A plurality of markers attached to the bucket and arranged on one straight line perpendicular to the rotation axis of the bucket;
At least two imaging means provided at different positions of the construction machine main body for imaging an image including the plurality of markers;
An image transmitter that is provided in the construction machine and wirelessly transmits an image captured by the imaging unit;
An image receiver that is provided independently of the construction machine and receives an image transmitted from the image transmitter;
A blade edge position calculator that is provided independently of the construction machine and that calculates the blade edge position of the bucket based on an image received by the image receiver;
The cutting edge position calculator is
By binarizing the image photographed by the photographing means, the plurality of markers are extracted and positions on the image are detected, and the positions of the plurality of markers are detected by triangulation from the detected positions on the image. Obtain a three-dimensional position, obtain the inclination of a vector connecting the plurality of markers from the obtained three-dimensional position of the plurality of markers, obtain the obtained three-dimensional position of the plurality of markers, obtain the obtained inclination of the vector, and in advance The blade edge position of the bucket is obtained from the offset value of the blade edge position with respect to one of the plurality of markers.

A cutting edge position measuring device for a construction machine according to a third invention that solves the above-described problem,
In the cutting edge position measuring device for a construction machine according to the first or second invention,
Instead of the binarization process, the blade edge position calculator performs template matching by performing correlation calculation with an image including the plurality of markers captured in advance by the imaging unit, thereby obtaining an image from the image captured by the imaging unit. The plurality of markers are extracted and positions on the image are detected.

A cutting edge position measuring device for a construction machine according to a fourth invention that solves the above-described problem,
In the cutting edge position measuring device for a construction machine according to the first or second invention,
The blade edge position calculator extracts the plurality of markers from the image photographed by the photographing means by performing color extraction suitable for the plurality of markers instead of the binarization processing, and positions on the image Is detected.

The present invention has the following effects.
(1) Compared with the measurement of the cutting edge position by the operator's eye measurement, the automatic measurement by the machine is performed, so that the cutting edge position can be measured without variation regardless of the skill level of the driving operator.
(2) Since the cutting edge position is digitized compared to the measurement of the cutting edge position by the operator's eye measurement, the cutting edge position data can be easily diverted to other devices.
(3) Compared to blade edge position measurement with a rotation angle sensor, only the camera and marker need be attached to the outside, and the attachment work is easy. (4) Compared to blade edge position measurement with a rotation angle sensor, the boom, arm, There is no need to find the length of a movable part such as a bucket. If the excavator to be used is changed, the adjustment work can be simplified because it is only necessary to attach the marker and measure the blade edge position from the marker.
(5) Compared to the measurement of the blade edge position by the rotation angle sensor, the blade edge position is obtained directly from the marker fixed to the bucket, so there is no accumulation of calculation position error due to looseness of movable joints such as booms, arms, and buckets. Accurate cutting edge position can be measured.
(6) Compared with the measurement of the blade edge position by the rotation angle sensor, the sensor failure rate is low because there is no sensor installed on the movable part, and the operation cost for maintenance can be kept small.

  Hereinafter, an embodiment of a blade edge position measuring device for a construction machine according to the present invention will be described with reference to the drawings.

  FIG. 1 is a schematic configuration diagram illustrating an example of an embodiment of a cutting edge position measuring device for a construction machine according to the present invention. FIG. 2 is a block diagram showing an example of an embodiment of a cutting edge position measuring device for a construction machine according to the present invention, and FIG. 3 is a flowchart showing an example of an embodiment of a cutting edge position measuring device for a construction machine according to the present invention. It is. In the following description, including the present embodiment, the cutting edge position measuring device for a construction machine according to the present invention will be described using a hydraulic excavator as an example of the construction machine.

  The hydraulic excavator 1 (construction machine) has a boom 2 attached to the main body, an arm 3 attached to the boom 2, and a bucket 4 attached to the arm 3. It is driven by a hydraulic cylinder that performs a predetermined operation. The position of the blade edge 5 of the bucket 4 is conventionally measured by an operator's eye measurement, measurement by a rotation angle sensor provided on the boom 2, arm 3, bucket 4, and the like. It was.

  Therefore, in the present embodiment, in order to measure the position of the blade edge 5, a plurality of markers M1, M2,... (Only two of M1 and M2 are shown as an example in FIG. 1). Install and fix. Further, at least two cameras (left camera 6L, right camera 6R) are installed on the main body of the hydraulic excavator 1 at a position where the markers M1, M2,... Can be seen (for example, on the roof of the driver's seat). Furthermore, a blade edge position calculator 7 for calculating the blade edge position from the images of the left camera 6L and the right camera 6R was installed. Here, when viewed from the driver's seat, the camera on the left side is called “left camera”, and the other camera on the right side is called “right camera”.

  In this blade edge position calculator 7, markers M1, M2,... Are extracted from images taken by the two cameras 6L, 6R, and the positions of the markers M1, M2,. The three-dimensional positions of the markers M1, M2,... Are obtained, and the position of the blade edge 5 of the excavator 1 is calculated from the obtained three-dimensional positions of the markers M1, M2,. Note that the markers are numbered M1, M2,... From the side closer to the blade edge 5. Further, as the markers M1, M2,..., A shape such as a spherical shape, a round plate shape, a round pattern on the rectangular plate, a checkered pattern on the rectangular plate, or a light emitting body is used.

  Then, for the bucket 4 to which the markers M1, M2,... Are attached, as shown in FIG. 1 (b), a coordinate system having the marker M1 closest to the blade edge 5 as the origin (hereinafter referred to as “bucket coordinate system Σb”). .). In the bucket coordinate system Σb, the movement of the bucket 4 in the rotational direction is defined to be constrained to the rotational direction with respect to the X axis. The markers M1, M2,... Are arranged on one straight line and on a YZ plane orthogonal to the X axis. That is, the markers M1, M2,... Are arranged on one straight line that is perpendicular to the X axis (that is, the rotation axis of the bucket 4). In the bucket coordinate system Σb, an offset value related to each coordinate axis from the origin (the position of the marker M1) to the cutting edge center 5c, that is, the position (x_off, y_off, z_off) of the cutting edge center 5c is obtained in advance.

  Here, with reference to the block diagram of FIG. 2 and the flowchart of FIG. 3, the processing procedure in the blade edge position calculator 7 will be described in detail.

(Step S1) Image input The left camera 6L and the right camera 6R attached to the hydraulic excavator 1 acquire an image of the work place including the markers M1, M2,... Attached to the bucket 4, and the acquired image is used as the blade edge position. The data is input to the memory 12 via the image input unit 11 of the computer 7 and recorded. In the blade edge position calculator 7, the marker data input unit 13 uses the color, shape, pattern, and luminance values on the partial images surrounding the markers M 1, M 2,. Each marker data such as is input to the memory 12 in advance. In the camera data input unit 14, camera data such as the positions and orientations of the cameras 6 </ b> R and 6 </ b> L, the CCD (Charge-Coupled Device) size, and the focal length of the lens are input to the memory 12 in advance. In addition, in the blade edge data input unit 15, blade edge data of the position (x_off, y_off, z_off) of the blade edge center 5 c in the bucket coordinate system Σb described above is input to the memory 12 in advance.

(Step S2) Marker Extraction The marker extraction unit 16a performs binarization processing on the image recorded in the memory 12, detects the positions of the markers M1, M2,... On the image, and stores the marker positions on the image in the memory 12. To record. For spherical, round, and illuminant markers, circles are extracted on the image, and for checkered markers, rectangles on the image are extracted to become markers. What is necessary is just to detect the position of a part.

(Step S3) Measurement of marker three-dimensional position In the marker three-dimensional position calculation unit 17, an image of the left camera 6L (hereinafter referred to as “left image”) and an image of the right camera 6R (hereinafter referred to as “right image”) recorded in the memory 12. 3), the three-dimensional positions of the markers M1, M2,... Are obtained from the marker positions on the image detected by the marker extraction unit 16a by triangulation, and the three-dimensional positions of the obtained markers M1, M2,. The position is recorded in the memory 12. At this time, the three-dimensional positions of the markers M1, M2,... Are obtained as positions (x1, y1, z1), (x2, y2, z2).

(Step S4) Calculation of Cutting Edge Position As described above, the position (x_off, y_off, z_off) of the cutting edge center 5c in the bucket coordinate system Σb is obtained in advance and input to the memory 12. Also, the marker three-dimensional position calculation unit 17 obtains the three-dimensional positions (x1, y1, z1), (x2, y2, z2),... Of the markers M1, M2,. Yes. In the blade edge position calculation unit 18, the position of the blade edge center 5c (x_off, y_off, z_off) in the bucket coordinate system Σb, the markers M1, M2,. Using the three-dimensional position (x1, y1, z1), (x2, y2, z2)... To perform the following coordinate transformation, the blade edge position (xe, ye, ze) is calculated, The calculated blade edge position (xe, ye, ze) is recorded in the memory 12.

  At this time, the coordinate transformation Tob from the object coordinate system Σo to the bucket coordinate system Σb is obtained from the position of the marker M1 that is the origin of the bucket coordinate system Σb for parallel movement, and the hydraulic excavator 1 for rotation. Since there is only rotation around the X axis, the rotation is determined from the rotation of the bucket 4 in the YZ plane. The rotation angle ψ about the X axis is obtained as an angle (tilt) formed by a vector from the marker M1 as the origin to another marker M2,.

For example, when there are two markers, the rotation angle ψ is obtained from the following equation using the markers M1 and M2.
ψ = −tan ⁻¹ {(y2−y1) / (z2−z1)}

Further, when there are three or more markers, the respective rotation angles are obtained from the vector from the marker M1 to the other markers M2,..., And the average thereof is defined as the rotation angle ψ. When N markers (N ≧ 3) are installed, it is obtained from the following equation.
ψ = − [tan ⁻¹ {(y2−y1) / (z2−z1)} +
tan ^-1 {(y3-y1) / (z3-z1)}
・
・
・
tan ⁻¹ {(yN−y1) / (zN−z1)}] / (N−1)

When the origin of the bucket coordinate system Σb is (xb, yb, zb) = (x1, y1, z1), the blade edge position (xe, ye, ze) can be calculated from the following equation.
xe = x_off + xb
ye = y_off · cosψ−z_off · sinψ + yb
ze = y_off · sinφ + z_off · cosψ + zb

  Then, the cutting edge position (xe, ye, ze) obtained by the above calculation and recorded in the memory 12 is output via the cutting edge position output unit 19.

  In this embodiment, since automatic measurement is performed by a machine, there is no variation in the measurement position as compared with the blade edge position measurement based on the driver's eye measurement regardless of the skill level of the operator. Further, since the blade tip position is digitized, it is easy to use the blade tip position data for other devices. Further, only the camera and the marker may be attached to the outside, and the attaching operation is easier than the blade edge position measurement by the rotation angle sensor. Further, it is not necessary to know the state of the movable parts such as the boom, arm, and bucket, and there is no accumulation of calculation position errors due to looseness of each joint. Furthermore, since there is no sensor attached to the movable part, it is possible to reduce the incidence of sensor failure.

  FIG. 4 is a block diagram showing another example of the embodiment of the cutting edge position measuring device for a construction machine according to the present invention.

  In the cutting edge position measurement apparatus of the present embodiment, only the marker extraction unit 16b in the cutting edge position calculator 7 is different from the marker extraction unit 16a in the cutting edge position calculator 7 of the cutting edge position measurement apparatus of Embodiment 1, and is replaced with the marker extraction unit 16a. Thus, the marker extraction unit 16b is used. Therefore, the description overlapping with the first embodiment is omitted, and the cutting edge position measuring apparatus of the present embodiment will be described.

  In the marker extraction unit 16a in the blade position calculator 7 of the blade position measurement device of the first embodiment, marker extraction is performed by binarization processing, but the marker extraction unit in the blade position calculator 7 of the blade position measurement device of the present embodiment. In 16b, marker extraction is performed by template matching.

  Specifically, an image of the marker placed at the reference position is captured in advance using the cameras 6R and 6L, and a partial image surrounding the marker on the image (hereinafter referred to as “marker template”). Acquired and recorded in the memory 12. Then, the marker extraction unit 16b detects a marker position on the input image by searching for a portion similar to the marker template on the newly input image by correlation calculation (template matching), The marker position on the image is recorded in the memory 12. Thereafter, the cutting edge position is calculated and output in the same procedure as in the first embodiment.

  In the present embodiment, in addition to the effect of the blade edge position measuring device shown in the first embodiment, template matching is performed based on the image information of the marker having a larger amount of information than the binarization process, and therefore, the effect of reducing erroneous marker detection is reduced. There is.

  FIG. 5 is a block diagram showing another example of the embodiment of the cutting edge position measuring device for a construction machine according to the present invention.

  In the cutting edge position measurement apparatus of the present embodiment, only the marker extraction unit 16c of the cutting edge position calculator 7 is different from the marker extraction units 16a and 16b in the cutting edge position calculator 7 of the cutting edge position measurement apparatus of Embodiments 1 and 2, and marker extraction is performed. Instead of the units 16a and 16b, a marker extracting unit 16c is used. Therefore, the description overlapping with the first and second embodiments will be omitted, and the blade edge position measuring apparatus of the present embodiment will be described.

  The marker extraction unit 16a in the blade position calculator 7 of the blade position measurement device of the first embodiment performs marker extraction by binarization processing, and the marker extraction unit 16b in the blade position calculator 7 of the blade position measurement device of the second embodiment. Then, marker extraction is performed by template matching, but marker extraction is performed by color extraction in the marker extraction unit 16c in the blade position calculator 7 of the blade position measurement device of the present embodiment.

  Specifically, marker color information is acquired in advance and recorded in the memory 12. Then, the marker extracting unit 16c detects a marker position on the input image by extracting a color portion that matches the marker from the newly input image, and records the marker position on the image in the memory 12. . Thereafter, the cutting edge position is calculated and output in the same procedure as in the first embodiment.

  In the present embodiment, in addition to the effect of the blade edge position measuring apparatus shown in the first embodiment, the marker is extracted by color extraction, so that the influence of these is suppressed even in a situation where various existing structures and natural objects are mixed. Marker extraction can be performed reliably.

  FIG. 6 is a schematic configuration diagram showing another example of the embodiment of the cutting edge position measuring device for a construction machine according to the present invention, and FIG. 7 shows another embodiment of the cutting edge position measuring device for the construction machine according to the present invention. It is a block diagram which shows an example.

  In the cutting edge position measuring apparatus of the present embodiment, the installation position of the cutting edge position calculator 7 is different from the cutting edge position measuring apparatus of the first to third embodiments. Specifically, in the cutting edge position measuring apparatus shown in the first to third embodiments, the cutting edge position calculator 7 is provided in the main body of the hydraulic excavator 1, whereas in the cutting edge position measuring apparatus of the present embodiment, the cutting edge position calculator 7 is provided. Instead of this, an image transmitter 8 that wirelessly transmits images acquired by the cameras 6R and 6L is provided in the main body of the hydraulic excavator 1, and an image receiver 9 that receives an image transmitted from the image transmitter 8, and an image receiver The blade position calculator 7 that receives the image received at 9 and calculates the blade position is provided independently on the remote terminal 10 at a location different from the main body of the hydraulic excavator 1.

  In addition, in the blade edge position calculator 7 in the blade edge position measuring apparatus of the present embodiment, the blade edge position is calculated by the same procedure as any one of the first to third embodiments described above. That is, the blade edge position is calculated by using any of the marker extraction units 16a, 16b, and 16c in the blade position calculator 7 of the first to third embodiments as the marker extraction unit 16 in the blade position calculator 7 of the present embodiment. Done. Therefore, the description of the blade tip position calculation procedure is omitted here.

  In this embodiment, in addition to the effects of the blade edge position measuring apparatus shown in Embodiments 1 to 3, the blade edge position calculator 7 is installed in the main body of the hydraulic excavator 1 by wirelessly acquiring an image necessary for calculating the blade edge position. Even when there is no room to do so, the cutting edge position of the excavator 1 can be calculated.

  The present invention is applicable to a construction machine such as a hydraulic excavator, and measures a blade edge position by processing images from a plurality of cameras mounted on the construction machine.

BRIEF DESCRIPTION OF THE DRAWINGS It is a schematic block diagram which shows an example (Example 1) of embodiment of the blade position measuring device of the construction machine which concerns on this invention, (a) shows the whole, (b) shows a bucket part. . It is a block diagram which shows an example (Example 1) of embodiment of the blade edge | tip position measuring device of the construction machine which concerns on this invention. It is a flowchart which shows an example (Example 1) of embodiment of the blade-tip position measuring apparatus of the construction machine which concerns on this invention. It is a block diagram which shows another example (Example 2) of embodiment of the blade edge | tip position measuring apparatus of the construction machine which concerns on this invention. It is a block diagram which shows another example (Example 3) of embodiment of the blade edge | tip position measuring apparatus of the construction machine which concerns on this invention. It is a schematic block diagram which shows another example (Example 4) of embodiment of the blade-tip position measuring apparatus of the construction machine which concerns on this invention. It is a block diagram which shows another example (Example 4) of embodiment of the blade-tip position measuring apparatus of the construction machine which concerns on this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Hydraulic excavator 4 Bucket 5 Cutting edge 6R, 6L Camera 7 Cutting edge position calculator 8 Image transmitter 9 Image receiver 11 Image input part 12 Memory 16a, 16b, 16c Marker extraction part 17 Marker three-dimensional position calculation part 18 Cutting edge position calculation part 19 Cutting edge position output section

Claims (4)

A blade position measuring device for a construction machine that measures the blade edge position of a bucket of a construction machine,
A plurality of markers attached to the bucket and arranged on one straight line perpendicular to the rotation axis of the bucket;
At least two imaging means provided at different positions of the construction machine main body for imaging an image including the plurality of markers;
A blade edge position calculator that calculates the blade edge position of the bucket based on the image provided by the construction machine and photographed by the photographing means;
The cutting edge position calculator is
By binarizing the image photographed by the photographing means, the plurality of markers are extracted and positions on the image are detected, and the positions of the plurality of markers are detected by triangulation from the detected positions on the image. Obtain a three-dimensional position, obtain the inclination of a vector connecting the plurality of markers from the obtained three-dimensional position of the plurality of markers, obtain the obtained three-dimensional position of the plurality of markers, obtain the obtained inclination of the vector, and in advance A blade edge position measuring device for a construction machine, wherein the blade edge position of the bucket is obtained from an offset value of the blade edge position with respect to one of the plurality of markers. A blade position measuring device for a construction machine that measures the blade edge position of a bucket of a construction machine,
A plurality of markers attached to the bucket and arranged on one straight line perpendicular to the rotation axis of the bucket;
At least two imaging means provided at different positions of the construction machine main body for imaging an image including the plurality of markers;
An image transmitter that is provided in the construction machine and wirelessly transmits an image captured by the imaging unit;
An image receiver that is provided independently of the construction machine and receives an image transmitted from the image transmitter;
A blade edge position calculator that is provided independently of the construction machine and that calculates the blade edge position of the bucket based on an image received by the image receiver;
The cutting edge position calculator is
By binarizing the image photographed by the photographing means, the plurality of markers are extracted and positions on the image are detected, and the positions of the plurality of markers are detected by triangulation from the detected positions on the image. Obtain a three-dimensional position, obtain the inclination of a vector connecting the plurality of markers from the obtained three-dimensional position of the plurality of markers, obtain the obtained three-dimensional position of the plurality of markers, obtain the obtained inclination of the vector, and in advance A blade edge position measuring device for a construction machine, wherein the blade edge position of the bucket is obtained from an offset value of the blade edge position with respect to one of the plurality of markers. In the cutting edge position measuring device of the construction machine according to claim 1 or claim 2,
Instead of the binarization process, the blade edge position calculator performs template matching by performing correlation calculation with an image including the plurality of markers captured in advance by the imaging unit, thereby obtaining an image from the image captured by the imaging unit. A blade edge position measuring apparatus for a construction machine that extracts the plurality of markers and detects a position on an image. In the cutting edge position measuring device of the construction machine according to claim 1 or claim 2,
The blade edge position calculator extracts the plurality of markers from the image photographed by the photographing means by performing color extraction suitable for the plurality of markers instead of the binarization processing, and positions on the image A blade edge position measuring device for a construction machine, wherein

Images