JP2008240362A - Construction machine and method of guiding backward movement of construction machine - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To enhance safety during backward movement by easily and properly performing a guidance during the backward movement by merely using one camera. <P>SOLUTION: A wide field camera 9 is installed at the rear of an upper structure 3 and can photograph both a crawler 2 and the rear part of a rear structure 3. Both the crawler 2 and the rear part of the upper structure 3 are photographed simultaneously, and the relative angle of the crawler 2 to the upper structure 3 is calculated from the wide view photographed by the camera 9. Therefore, the relative angle α of the crawler 2 to the upper structure 3 can be measured by merely using one wide field camera 9 without adding a revolving angle sensor. Also, the advancing direction of the hydraulic shovel 1 during the backward movement is predicted based on the calculated relative angle α, and the rear image photographed by the camera 9 and the image indicating the predicted advancing direction are displayed on a monitor in a superimposed state. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a construction machine having a crawler belt such as a hydraulic excavator and a turning body, and a backward movement guidance method for the construction machine.

  Conventionally, regarding a construction machine such as a hydraulic excavator, the rear side of the construction machine is photographed with a camera, and the photographed rear view image can be displayed on the monitor, but the image displayed on the monitor is switched by the operation of the machine. There is a proposed example (see, for example, Patent Document 1).

  On the other hand, with regard to automobiles, there is a parking guidance that displays a predicted path when the automobile moves backward when a rear image taken by a rear camera is displayed on a back monitor, and has been put into practical use.

JP 2006-336275 A

  However, the one described in Patent Document 1 merely allows a rear view image to be displayed on a monitor, and the traveling direction predicted when the construction machine is going backward is not known from the monitor screen, and is used as backward movement guidance. It is insufficient.

  In this regard, according to the parking guidance in the case of an automobile, the predicted course at the time of reverse travel is also displayed, so that safety during reverse travel can be ensured. However, in the case of an automobile, the traveling direction during backward movement of the automobile can be predicted only by the steering angle. However, a construction machine such as a hydraulic excavator has an upper turning body on the crawler track, so The direction does not always coincide with the direction of travel during backward movement, and in order to predict the direction of travel, not only the operation information of the travel lever but also the information on the relative positional relationship between the upper turning body and the crawler belt is necessary. The parking guidance technology cannot be applied as it is.

  Here, the information on the relative positional relationship between the upper-part turning body and the crawler belt can be measured by adding a turning angle sensor such as an encoder or a potentiometer, but this increases the cost. In addition to the camera that captures the rear field of view, a camera for measuring the relative angle between the crawler belt and the upper swinging body is provided to determine the relative angle between the crawler belt and the upper swinging body to predict the traveling direction, Although it is conceivable to superimpose on the field-of-view image, in general, when a plurality of cameras are used, calibration between the cameras is required, and it is difficult to accurately obtain the angle. As a result, the guidance in the traveling direction is not appropriate.

  The present invention has been made in view of the above, and it is possible to easily and appropriately provide guidance during backward movement by using only one camera, and to improve safety during backward movement. It is an object of the present invention to provide a construction machine capable of performing the same and a backward movement guidance method for the construction machine.

  In order to solve the above-described problems and achieve the object, a backward movement guidance method for a construction machine according to the present invention is a backward movement guidance method for a construction machine having a crawler belt and a turning body, and the rear side of the turning body A step of photographing with a wide-viewing angle camera that is installed in the camera and capable of photographing the crawler belt and the rear of the revolving body, and a step of calculating a relative angle between the crawler belt and the revolving body from an image photographed by the camera And a backward traveling direction of the construction machine is predicted based on the calculated relative angle, and a rear image photographed by the camera and an image indicating the predicted traveling direction are mounted on the revolving structure. And a step of displaying the image superimposed on a monitor.

  In the backward movement guidance method for a construction machine according to the present invention as set forth in the invention described above, the traveling direction is predicted by taking into account the operation content of the operation lever.

  In the backward movement guidance method for a construction machine according to the present invention, in the above invention, in the displaying step, the crawler belt width through which the crawler belt passes is displayed on the monitor in addition to the image indicating the traveling direction. And

  The construction machine according to the present invention is a construction machine having a crawler belt and a swivel body, and is a camera with a wide viewing angle that is installed on the rear side of the swivel body and can photograph the crawler belt and the rear of the swivel body. A monitor mounted on the revolving structure, computing means for computing a relative angle between the crawler belt and the revolving structure from an image taken by the camera, and a reverse of the construction machine based on the computed relative angle Prediction means for predicting the traveling direction at the time, and a backward movement guidance display means for displaying a rear image captured by the camera and an image indicating the predicted traveling direction in a superimposed manner on the monitor. It is characterized by.

  Moreover, the construction machine according to the present invention is characterized in that, in the above invention, the prediction means predicts the traveling direction of the construction machine when the construction machine moves backward in consideration of the operation content of the operation lever.

  In the construction machine according to the present invention as set forth in the invention described above, the backward movement guidance display means displays on the monitor a crawler belt width through which the crawler belt passes in addition to an image indicating a traveling direction.

  The construction machine and the backward movement guidance method for the construction machine according to the present invention use a wide-viewing angle camera that is installed on the rear side of the revolving structure and can photograph the crawler track and the back of the revolving structure. And the relative angle between the crawler belt and the swivel body is calculated from the image captured by the camera, and the traveling direction of the construction machine when moving backward is predicted based on the calculated relative angle. Without adding, it is possible to measure the relative angle between the crawler track and the swivel body by using only one wide viewing angle camera, and the rear image captured by the camera and the image indicating the predicted traveling direction Are superimposed and displayed on the monitor, so that the guidance during the backward movement can be properly performed, and the safety during the backward movement can be improved.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. The present embodiment shows an application example to a hydraulic excavator as a construction machine. The present invention is not limited to the embodiment, and various modifications can be made without departing from the spirit of the present invention.

  FIG. 1 is a schematic side view showing a configuration example of a hydraulic excavator to which the backward movement guidance method of the present embodiment is applied, and FIG. 2 is a schematic block diagram showing a configuration example of the hydraulic excavator. The hydraulic excavator 1 according to the present embodiment includes an upper swing body 3 including a cockpit 3a and the like on a pair of left and right crawler tracks 2 forming a lower traveling body, and a vehicle body portion of the upper swing body 3 includes a boom 4 and an arm 5. And the working machine 7 which consists of the bucket 6 is attached. Although not shown in FIG. 1, the boom 4, arm 5, and bucket 6 are actuated by driving hydraulic actuators 8 (see FIG. 2) such as hydraulic cylinders for booms, arms, and buckets. Perform the desired work. Further, the upper swing body 3 is rotationally moved in the horizontal plane with respect to the crawler belt 2 when the swing motor 8a in the hydraulic actuator 8 is driven, so that the direction of the work implement 7 can be set to an arbitrary direction. Further, the pair of left and right crawler belts 2 are operated by driving the right traveling motor 8b and the left traveling motor 8c in the hydraulic actuator 8, and move forward or backward while moving straight or turning.

  Here, the hydraulic excavator 1 of the present embodiment includes a camera 9 with the photographing direction facing downward at a position on the rear side of the upper swing body 3, for example, a lower position of the counterweight 3b. The camera 9 photographs a subject using a solid-state imaging device such as a CCD or CMOS sensor. In the present embodiment, as shown by the line of sight in FIG. A wide viewing angle camera capable of simultaneously photographing a predetermined range of 3 is used. As the wide viewing angle camera, an omnidirectional camera, a fisheye camera, or the like can be applied. In this embodiment, an omnidirectional camera using a hyperbolic mirror is used as the camera 9.

  Further, as shown in FIG. 2, the excavator 1 of the present embodiment includes a vehicle body controller 10 and an image processing controller 20 as a control system, and is connected by a communication bus 31. The vehicle body controller 10 controls most of the hydraulic excavator 1. For example, the vehicle body controller 10 controls the engine-driven hydraulic pump 12 by controlling the number of revolutions of the engine 11 with the fuel injection command value, and via the operation valve 13. The hydraulic actuator 8 such as a hydraulic cylinder, a hydraulic motor (a turning motor 8a, a right traveling motor 8b, and a left traveling motor 8c) is driven and controlled to perform a desired operation and operation. Further, the vehicle body controller 10 takes in an operation lever signal of the operation lever 14 for driving the corresponding hydraulic actuator 8 by controlling the operation valve 13 with the pilot oil pressure, and a fuel injection command value corresponding to the operation amount of the operation lever 14. Is calculated. Here, the operation lever 14 includes a turning lever 14a, a right traveling lever 14b, a left traveling lever 14c, a boom operating lever, an arm operating lever, a bucket operating lever, etc. (not shown).

  The image processing controller 20 is for capturing image information captured by the camera 9 and performing image processing for backward movement guidance during back-monitoring when the excavator 1 moves backward. In addition, a monitor 21, a back monitor display button 22, a conversion table 23, and the like are connected. The monitor 21 is for a back monitor mounted at, for example, a front position in the cockpit 3a, and the back monitor display button 22 is for instructing the monitor display when the operator needs to display the monitor 21. . The image processing controller 20 operates when a monitor display is instructed by the back monitor display button 22 and realizes functions such as a calculation means, a prediction means, and a backward movement guidance display means.

  An example of operation control of the image processing controller 20 of the present embodiment will be described with reference to a schematic flowchart shown in FIG. First, when monitor display is instructed by the back monitor display button 22, the camera 9 performs a photographing operation to photograph a wide range including a part of the rear side of the crawler belt 2 and the rear of the upper swing body 3, and an image processing controller. 20 is output. FIG. 4 schematically shows an example of a wide-angle image 40 taken by the camera 9, and includes a crawler belt image 41 and a rear image 42.

  When the wide-angle image 40 is input from the camera 9 (step S1), the image processing controller 20 extracts the crawler belt region from the wide-angle image 40 by image processing such as feature point extraction (step S2). Each edge portion is extracted as an edge line segment 51 as indicated by a broken line in FIG. 5 (step S3). Then, as shown in FIG. 5, the inclination of the crawler belt 2 on the photographing screen coordinates of each edge line segment 51 of the crawler belt area is obtained, and the average is taken to calculate the inclination θ of the crawler belt 2 on the screen. (Step S4).

  Next, the relative angle between the crawler belt 2 and the upper swing body 3 is calculated (step S5). That is, since the camera 9 is fixed to a part of the upper swing body 3, the position of the upper swing body 3 is fixed on the photographing screen, and the relative angle α between the actual upper swing body 3 and the crawler belt 2 is fixed. Can be obtained by referring to the conversion table 23 that defines the correspondence with the inclination θ of the crawler belt 2 on the screen. In the conversion table 23, the correspondence between the inclination angle θ of the crawler belt 2 on the photographing screen shown in FIG. 6A and the relative angle α between the actual upper swing body 3 and the crawler belt 2 shown in FIG. The relationship is defined in advance at the time of design. For example, a correspondence relationship as shown in FIG. 7 is set.

On the other hand, in parallel with the above-described image processing, travel lever input values relating to the right travel lever 14b and the left travel lever 14c in the operation lever 14 during backward movement are acquired (step S11). The traveling lever value may be a pilot hydraulic pressure value generated by operating the operation lever 14 or an electric lever input signal accompanying the lever operation. Then, taking into account the lever input values (operation details) of the right and left right travel levers 14b and 14c, the traveling direction in the coordinate system viewed from the direction of the crawler belt 2 is estimated as shown in FIG. As shown in FIG. In the example shown in FIG. 8, assuming that the radius of the driving wheel of the crawler belt 2 is r, the angular velocities of the right and left driving wheels are dθ _R / dt, dθ _L / dt, and the distance between the left and right crawler belts 2 is 2a. The following equation 1 is obtained.

  Here, if the lever input values of the right traveling lever 14b and the left traveling lever 14c are the same, the predicted course 61 is a straight traveling direction along the direction of the crawler belt 2, but the right traveling lever 14b and the left traveling lever 14c are levers. When the input values are different, the direction is such that the vehicle turns to the right or to the left according to the magnitude relationship.

  Next, as shown in FIG. 9 (b), the predicted course 61 calculated in step S12 as shown in FIG. 9A is equivalent to the relative angle α between the crawler belt 2 and the upper swing body 3 obtained in step S5. By rotating, as shown in FIG.9 (c), it converts into the predicted course 61 seen from the upper turning body 3 (step S21).

On the other hand, in parallel with these image processing and course prediction processing, the wide angle image 40 photographed and input by the camera 9 is converted into a rear viewpoint image in order to display the rear image 42 portion on the monitor 21. This is performed (step S31). That is, the wide-angle image 40 captured by the camera 9 is a surrounding panoramic image including an omnidirectional image as shown in FIG. 10A, and this is panorama developed into a panoramic image as shown in FIG. 10B. Then, a rear display image 44 to be displayed on the monitor 21 is created by cutting out a portion corresponding to the rear monitoring area 43 including the desired rear image 42 from the developed panoramic image. At this time, if the radius of the omnidirectional wide-angle image 40 is r, an arbitrary point on the wide-angle image 40 is (x, y), and the center coordinates are (x ₀ , y ₀ ), The corresponding point (X, Y) in
X = rtan ⁻¹ {(y ₀ −y) / (x ₀ −x)}
Y = r−√ {(x ₀ −x) ² + (y ₀ −y) ² }
Is calculated by

  Next, in step S21, the predicted course 61 converted into the coordinate system of the upper swing body 3 is perspectively projected on the rear viewpoint coordinate system by a known perspective projection method assuming that the ground is a plane (step S22). That is, in the XYZ coordinate system as shown in FIG. 11, the function f (X, Y) of the predicted path 61 expressed on the XY surface corresponding to the ground is seen on the XZ plane corresponding to the screen of the monitor 21, for example. This is obtained as a projected function g (x, y).

This process is processed by a perspective projection method. First, the gazing point P _w (x _w , y _w , z _w ) defined in the world coordinate system as shown in FIG. 12 is converted into a viewpoint P _f (x _f , y _f , z _f ) in the viewpoint coordinate system. . Here, in the case of FIG.
a = √ {(x _f −x _a ) ² + (y _f −y _a ) ² }
^{b = √ {a 2 + (} z f -z a) 2}
sin θ = (y _f −y _a ) / a
cos θ = (x _f −x _a ) / a
_{sinφ = (z f -z a)} / b
cosφ = a / b
In this case, it is converted into the viewpoint coordinate system by the following formula 2.

The coordinates of the viewpoint coordinate system obtained in this way are
x _a ′ = x _a / z _a
y _a ′ = y _a / z _a
Is converted into coordinates (x _a ′, y _a ′) on the screen of the monitor 21 using the following conversion formula.

  Finally, when the rear display image 44 as shown in FIG. 13A is displayed on the screen of the monitor 21, the guidance path 61 is an image indicating the traveling direction of the predicted course 61 thus coordinate-converted. As shown in FIG. 13B, the line 45 is overlaid and displayed as an overlay (step S41). At this time, as shown in FIG. 13B, using the information of the interval 2a between the crawler belts 2, an image like a broken line indicating the crawler belt width that the crawler belt 2 is predicted to pass as it progresses in the traveling direction. 46 is displayed simultaneously.

  As described above, according to the present embodiment, the crawler belt 2 and the upper portion of the upper swivel body 3 are used by using the wide viewing angle camera 9 installed on the rear side of the upper revolving body 3 and capable of photographing the crawler belt 2 and the rear of the rear swivel body 3. Since the rear of the revolving structure 3 is simultaneously photographed by the camera 9, and the relative angle α between the crawler belt 2 and the upper revolving structure 3 is calculated from the wide-angle image 40 photographed by the camera 9, a turning angle sensor is added. Therefore, it is possible to measure the relative angle α between the crawler belt 2 and the upper swing body 3 by using only one camera 9 having a wide viewing angle, and when the hydraulic excavator 1 moves backward based on the calculated relative angle α. The rear display image 44 based on the rear image 42 photographed by the camera 9 and the guidance line 45 that is an image indicating the predicted travel direction are superimposed and displayed on the monitor 21. The operator has controlled the cockpit 3a The moving direction with respect to the rearward image 42 plainly when trying to reverse movement while watching the screen of the monitor 21, it is possible to properly perform the guidance at the time of backward movement, it is possible to improve the safety during backward movement. In particular, in addition to the display of the guidance line 45 indicating the traveling direction, an image 46 indicating the width of the crawler belt that the crawler belt 2 is predicted to pass is also displayed, so that the operator can grasp the presence or absence of an object in the danger range when moving backward. It is easy to do and can improve safety | security further.

  In the present embodiment, when the back monitor display button 22 is pressed by the operator, the backward movement guidance is displayed at a desired timing prior to the backward movement by performing the process for the backward movement guidance. However, the process for the backward movement guidance may be performed from the time when the backward movement operation is performed by the right traveling lever 14b or the left traveling lever 14c.

  In the present embodiment, the traveling direction of the hydraulic excavator 1 is predicted based on the operation content of the operation lever 14, but the operation lever 14 is operated as in the case of the operation of the back monitor button 22 prior to the backward movement. If it is not time point, the traveling direction may be predicted by the direction of the relative angle α, assuming that the lever input values of the right traveling lever 14b and the left traveling lever 14c are the same (directly traveling).

  Furthermore, the construction machine is not limited to the hydraulic excavator 1 and can be similarly applied as long as it includes a crawler belt and a turning body. The hydraulic excavator may also be a hybrid type including a generator motor and a capacitor.

It is a schematic side view which shows the structural example of the hydraulic shovel to which the backward movement guidance method of embodiment of this invention is applied. It is a schematic block diagram which shows the structural example of a hydraulic shovel. It is a schematic flowchart which shows the process control example by an image processing controller. It is a figure which shows typically an example of the wide-angle image image | photographed with the camera. It is explanatory drawing which shows the example of extraction of an edge line segment of a crawler belt, and an inclination angle. It is explanatory drawing for calculating | requiring the relative angle of a crawler belt and an upper turning body. It is explanatory drawing which shows the content of the conversion table. It is explanatory drawing for calculating the advancing direction of a crawler belt. It is explanatory drawing which shows the conversion process to the predicted course seen from the upper turning body. It is explanatory drawing which shows the conversion process from the omnidirectional wide-angle image to the panoramic developed panoramic image. It is explanatory drawing which shows the example of a perspective projection process on the monitor screen of a predicted course. It is a figure for demonstrating the example of a conversion process from a world coordinate system to a viewpoint coordinate system. It is explanatory drawing which shows the example of an overlay display process of the guidance line to a back display image.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Hydraulic excavator 2 Crawler track 3 Upper turning body 9 Camera 14 Operation lever 21 Monitor

Claims (6)

A reverse movement guidance method for a construction machine having a crawler belt and a swivel body,
The step of photographing with a wide viewing angle camera that is installed on the rear side of the revolving unit and can photograph the crawler belt and the back of the revolving unit;
Calculating a relative angle between the crawler belt and the swivel body from an image captured by the camera;
Based on the calculated relative angle, a forward traveling direction of the construction machine is predicted, and a rear image photographed by the camera and an image indicating the predicted traveling direction are displayed on a monitor mounted on the revolving structure. A process of displaying the image superimposed on
A backward movement guidance method for a construction machine, comprising:   2. The backward movement guidance method for a construction machine according to claim 1, wherein the advancing direction is predicted in consideration of an operation content of an operation lever.   3. The backward movement guidance method for a construction machine according to claim 1, wherein in the displaying step, a crawler belt width through which the crawler belt passes is displayed on the monitor in addition to an image indicating a traveling direction. A construction machine having a crawler belt and a swivel body,
A wide viewing angle camera installed on the rear side of the swivel body and capable of photographing the crawler belt and the back of the swivel body;
A monitor mounted on the revolving structure;
A computing means for computing a relative angle between the crawler belt and the swivel body from an image photographed by the camera;
Predicting means for predicting the traveling direction of the construction machine when traveling backward based on the calculated relative angle;
A backward movement guidance display means for displaying a rear image captured by the camera and an image indicating a predicted traveling direction in a superimposed manner on the monitor;
Construction machine characterized by comprising.   5. The construction machine according to claim 4, wherein the predicting unit predicts a traveling direction of the construction machine when traveling backward in consideration of an operation content of the operation lever.   The construction machine according to claim 4 or 5, wherein the reverse movement guidance display means displays on the monitor a crawler belt width through which the crawler belt passes in addition to an image indicating a traveling direction.

Images