JP2001348914A - Posture recognizing device for heavy machinery - Google Patents

Abstract

PROBLEM TO BE SOLVED: To solve the point at issue that monitor and control by an expert have been required for determining whether or not heavy machinery is positioned in a safe posture even in the case of both remote control and normal control, there has been possibility that even the expert mistakes a decision as to whether or not the heavy machinery is positioned in the safe posture and a beginner has been difficult to decide whether or not the heavy machinery is positioned in the safe posture in the case of monitor and control. SOLUTION: In the posture recognizing device for the heavy machinery, a sensor detecting longitudinal and horizontal inclinations under the horizontal state of the heavy machinery and a photographing means photographing operating conditions by the heavy machinery are installed to the heavy machinery while the posture-information display images 30, 40 of the heavy machinery based on a signal from the sensor and a photographing image 15 from the photographing means are displayed on a display means 5.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an apparatus for recognizing a posture of a heavy equipment such as a hydraulic shovel, a bulldozer, a dump truck, etc.

[0002]

2. Description of the Related Art In unmanned construction using heavy equipment remotely controlled in a hazardous area, it is difficult to remotely control the heavy equipment while grasping whether the posture is safe. Conventionally, for example, a heavy equipment such as a hydraulic shovel is equipped with a camera for photographing the front of the heavy equipment, and a work status of the heavy equipment photographed by the camera is transmitted to a remote control side, and a remote control is performed while watching the photographed image. I was trying to steer. However, in this case, when the heavy equipment is tilted, the camera is also tilted, so that it is impossible to determine from the captured image that the heavy equipment is tilted, and in the worst case, the heavy equipment may fall down. Therefore, by installing a camera outside the heavy equipment and monitoring the posture of the heavy equipment while working while watching the video taken from this external camera, it is determined whether the heavy equipment is in a safe posture I am trying to do it.

[0003]

However, in a remote operation, according to the method of installing a camera outside the heavy equipment and judging whether or not the heavy equipment is in a safe posture while watching a photographed image from the external camera, There is a limit to the zoom of the camera.Therefore, it is necessary to install the camera at a distance that can acquire the video that can grasp the inclination of the heavy equipment, and it is necessary to install the camera horizontally. But it has become increasingly difficult to establish such a place in a hazardous area. That is, the distance from the heavy equipment is restricted, and the setting of the camera becomes complicated. In addition, a skilled observer of the video from an external camera can see and judge intuitively based on his / her experience.
There is a possibility of misjudging whether the heavy equipment is in a safe position, and it is difficult to judge whether the heavy equipment is in a safe position based on images from external cameras if the observer is a beginner It is. In addition, even in a normal operation in which the operator gets on the driver's seat of the heavy equipment, a skilled operator can adjust the inclination of the heavy equipment in a safe range by the direction of gravity that is unconsciously recognized. In addition, there is a possibility that the judgment may be erroneous, and when the operator is a beginner, it is difficult to judge whether the heavy machine is in a safe posture and adjust the inclination of the heavy machine within a safe range.

Therefore, conventionally, in both remote operation and normal operation, it is necessary to monitor and operate the heavy equipment in order to grasp whether the heavy equipment is in a safe posture. May be misled in determining whether the vehicle is in a safe position, and in the case of novice surveillance and piloting,
There was a problem that it was difficult to determine whether the heavy equipment had a safe posture.

[0005]

An apparatus for recognizing a posture of a heavy equipment according to the present invention is provided with a sensor for detecting inclination of the heavy equipment in front, rear, left and right directions, and a photographing means for photographing a working condition of the heavy equipment. The display means provided in a moving operation room for remotely controlling the heavy equipment displays attitude information of the heavy equipment based on a signal from the sensor and photographing information from the photographing means. Further, the posture information and the photographing information are simultaneously displayed on the screen of one display means. Further, a sensor for detecting the inclination of the heavy machine in the front-rear direction and the left-right direction with respect to the horizontal state is attached to the heavy machine, and posture information of the heavy machine based on a signal from the sensor is displayed on display means provided in the operation room of the heavy machine. I did it. Further, in a heavy machine separated into an upper revolving unit and a lower traveling unit, the above sensors are attached to the upper revolving unit and the lower traveling unit, respectively, and the respective attitude information based on signals from these sensors is displayed on the display means. did. Further, in the above, when it is determined from the signal from the sensor that the inclination of the heavy equipment has deviated from the safe range, an alarm processing by voice or a visual alarm processing such as changing a color or form of display of the attitude information is performed. Is provided.

[0006]

DESCRIPTION OF THE PREFERRED EMBODIMENTS Embodiment 1 Hereinafter, a first embodiment of the present invention will be described with reference to the drawings. In FIG. 1, reference numeral 1 denotes a hydraulic shovel, which is a heavy machine remotely controlled in a dangerous area X, and reference numeral 2 denotes a mobile operation vehicle for remotely controlling the hydraulic shovel 1. In the case of a heavy machine that separates into an upper revolving unit 1A and a lower traveling unit 1B like the hydraulic excavator 1,
As shown in FIG. 2, it is preferable to install an inclination sensor on each of the upper swing body 1A and the lower traveling body 1B. The tilt sensor 3A measures the front and rear tilt angles (pitch angle) and left and right tilt angles (roll angle) of the upper swing body 1A with respect to the horizontal state of the upper swing body 1A.
Is for measuring the front and rear inclination angles (pitch angles) and left and right inclination angles (roll angles) of the lower traveling body 1B with respect to the horizontal state of the lower traveling body 1B. These tilt sensors 3
A and 3B may be sensors that use the gyro, sensors that use changes in the electric charge load or the magnetic field, etc., as long as they can detect the inclination of the excavator 1 with respect to the horizontal state of the excavator 1 before, after, and right and left. The tilt sensors 3A and 3B
When the hydraulic excavator 1 is attached to the upper swing body 1A and the lower traveling body 1B, calibration is set in advance in a horizontal state. Further, in the case of the hydraulic excavator 1, a CCD camera 4A for photographing the movement of the boom 1a, the working arm 1b and the bucket 1c, and a CCD camera 4B for photographing the foot and the place to be excavated by the bucket 1c are mounted.

As shown in FIG. 3, an upper revolving unit 1A based on a pitch angle and roll angle measurement signal measured by the tilt sensor 3A is provided in a moving operation room 20 of the moving operation vehicle 2.
, A posture information display image 40 of the undercarriage 1B based on the pitch and roll angle measurement signals measured by the inclination sensor 3B, and the inclination sensors 3A,
A posture information display image 50 indicating the rotation state of the upper revolving unit 1A with respect to the lower traveling unit 1B created based on the rotation angle obtained by calculating the inclination sensor data from 3B, and photographing from the CCD cameras 4A and 4B. Display means 5 is provided for displaying a combined image obtained by combining the image 15 and the image 15 on the operation TV screen 5f. Reference numeral 21 denotes an operation device for remotely controlling the excavator 1 while watching the operation TV screen 5f.

As shown in FIG. 4, a hydraulic excavator 1 includes a pitch angle and a roll angle measurement signals measured by the inclination sensors 3A and 3B, and C signals photographed by the CCD cameras 4A and 4B.
An AD conversion unit 6 as a signal processing device for wirelessly transmitting a CD camera image signal to the mobile operation vehicle 2;
A wireless transmitter 7 is provided. Thereby, the four analog signals of the pitch angle and the roll angle measured by the tilt sensors 3A and 3B are converted into one digital signal by the AD conversion unit 6 and sent to the wireless transmitter 7 as tilt sensor data. The data is combined with the CCD camera image signal captured by the CCD cameras 4A and 4B and transmitted to the mobile operation vehicle 2.

In the mobile operation room 20, data transmitted from the wireless transmitter 7 is transmitted to the wireless receiving antenna 2a.
(See FIG. 1) and receive the tilt sensor data and C
A radio receiver 8 that decomposes the image into a CD camera image signal, and an inclination sensor data sent from the radio receiver 8 are operated to determine the rotation angles of the upper revolving unit 1A and the lower traveling unit 1B to obtain an operation TV screen. A tilt data calculation unit 9 for generating the attitude information display image 50 to be displayed on the display means 5f, and generating the attitude information display images 30 and 40 for display on the operation TV screen 5f of the display means 5 based on the tilt sensor data; And an image synthesizing unit 10 for synthesizing the posture information display images 30, 40, and 50 with the CCD camera images photographed by the CCD cameras 4A and 4B.
As shown in the figure, the CCD camera 4 is displayed on the operation TV screen 5f.
A synthesized image obtained by synthesizing the posture information display images 30, 40, and 50 is displayed in the captured image 15 from A and 4B.

The tilt data calculation unit 9 and the image synthesis unit 10 are constituted by a personal computer or the like having an arithmetic processing program and a synthesis processing program.

By operating a foot switch or the like provided at the foot of the operating device 21 in the moving operation room 20,
Images captured by the CCD camera 4A and the CCD camera 4B can be switched and displayed.

As shown in FIG. 3, each of the attitude information display images 30 and 40 is an image in which an attitude information line is displayed in a circular frame R in which a vertical and horizontal reference line of a cross is set, and these are just images of an airplane. It is an image close to an image like an operation panel. The posture information display image 50 is an image in which the rotation state of the upper swing body 1A of the excavator 1 as viewed from above is displayed in the circle R as a pictorial diagram 51 of the excavator 1. In each of the posture information display images 30 and 40, the horizontal reference line A is a line indicating the pitch angle of 0 ° and the roll angle of 0 °, and the vertical reference line B is a line indicating the roll angle of 90 ° and is marked with the pitch angle scale. is there. In the posture information display image 50, although not shown, a scale is attached to the round frame R. That is, a position corresponding to the upper end of the vertical reference line B1 is a rotation angle of 0 °, a position corresponding to the lower end is a rotation angle of 180 °, a position corresponding to the left end of the horizontal reference line A1 is a rotation angle of 90 ° (or 270 °), At the position corresponding to the right end, graduations indicating a rotation angle of 270 ° (or 90 °) are provided, and graduations at intervals of the rotation angle of 15 ° are provided between them.

A posture information line (upper horizontal line) in each posture information display image 30, 40 indicates a pitch angle display line C1 indicating the current pitch angle of the upper swing body 1A, and a current pitch angle of the lower traveling body 1B. FIG. 3 shows a pitch angle display line C2.
Indicates that the upper revolving unit 1A and the lower traveling unit 1B are tilted upward. That is, the pitch angle display lines C1,
When C2 is displayed above the horizontal reference line A, it indicates that it is tilted forward. When the pitch angle display lines C1 and C2 are displayed below the horizontal reference line A, it is lowered (backward). Indicates that it is leaning. Although not shown, the vertical reference line B is provided with a scale indicating a pitch angle, and the upper end of the vertical reference line B indicates a forward rising pitch angle of 90 °, and the lower end of the vertical reference line B moves forward (upward). A pitch angle of 90 ° is shown, and graduations at intervals of, for example, a pitch angle of 15 ° are provided between the upper and lower ends of the vertical reference line B from the position of the pitch angle 0 ° which is the intersection (center) of the vertical reference line B and the horizontal reference line A. Is attached.

A posture information line (hatched line) in each posture information display image 30, 40 is a roll angle display line D1 indicating the current roll angle of the upper swing body 1A, and the current lower traveling body 1B.
In FIG. 3, the upper revolving unit 1A and the lower traveling unit 1B are inclined upward to the right. That is, the roll angle display lines D1, D2
Is inclined upward to the right through the intersection (center) between the vertical reference line B and the horizontal reference line A, indicating that the display is inclined to the right. Indicates that it is leaning. Although not shown, a scale representing the roll angle is provided around the round frame R. That is, the position corresponding to both ends of the horizontal reference line A is a roll angle of 0 °, and the position corresponding to the upper and lower ends of the vertical reference line B is a roll angle of 9 °.
At 0 °, graduations at intervals of, for example, a roll angle of 15 ° are provided.

The pitch angle display line, roll display line,
The vertical and horizontal reference lines may be displayed in different colors or in different forms such as chain lines and dotted lines.

Further, numerical values of the pitch angle and the roll angle may be displayed near the posture information display images 30 and 40, respectively. Further, a numerical value of the rotation angle may be displayed near the posture information display image 50.

Further, each posture information display image 30, 40, 5
0 is not necessary while the excavator 1 is performing a stable excavation operation in a stopped state. Therefore, although not shown, for example, each posture information display image 30,
It is preferable to provide a button or the like for turning on / off the display of the 40, 50 operation TV screen 5f.

According to the first embodiment, the upper revolving unit 1A and the lower traveling unit 1B of the hydraulically controlled excavator 1 are remotely controlled.
The tilt sensors 3A and 3B for detecting the inclination (pitch angle, roll angle) before, after, and left and right, and the CCD cameras 4A and 4B for photographing the working condition of the excavator 1 are mounted on the excavator 1 and Display means 5 provided in a mobile operation room 20 for remotely controlling the shovel 1
On the operation TV screen 5f, the posture information display images 30, 40 as the posture information of the upper revolving unit 1A and the lower traveling unit 1B of the excavator 1 based on the signals from the inclination sensors 3A, 3B, the pitch angle, and the like. Since the numerical information of the roll angle and the photographed images 15 from the CCD cameras 4A and 4B as the photographing information are combined and displayed, the upper revolving unit 1A and the lower traveling unit 1B of the excavator 1 are roughly approximated. Before, after, and left and right can be grasped. Further, the rotation status of the upper swing body 1A with respect to the lower running body 1B can be grasped from the posture information display image 50. Therefore, in remote control of the excavator 1, it is easy for a beginner as well as a skilled person to easily determine whether the excavator 1 is in a safe posture. Also, one display means 5
Each of the posture information display images 3 is displayed on the operation TV screen 5f.
Since 0, 40, and 50 and the photographed image 15 are displayed at the same time, it is possible to sequentially confirm whether the hydraulic shovel is in a safe posture while focusing on remote control without moving the line of sight extremely. In addition, since the posture information display images 30, 40, and 50 are displayed as posture information in which the inclination can be easily grasped instantaneously as compared with the numerical display, the visibility can be improved (that is, in the case of only the numerical display, the inclination is small). In the case of rising front and falling front, and in the case of rising right and falling right, the same numerical value must be changed in +-or color, etc., to grasp the inclination compared to the information display images 30, 40, etc. It takes time and visibility is not good.) Furthermore, since a numerical display is also performed, a specific angle can be confirmed. In the case of the present embodiment, since the image is displayed together with the posture information display images 30 and 40, the numerical value display may be simply a number.

It should be noted that the attitude information is not an attitude information display image 30, 40 as an image of an operation panel of an airplane as described above, but an attitude that makes it easy to grasp the inclination instantly as compared with a numerical display. As long as it is an information display image, the mode does not matter. Further, the posture information display image 50 does not necessarily have to be displayed.

Further, the posture information and the photographing information may be separately processed and displayed on the screens of different display means.

In the above description, an inclination sensor is installed on each of the upper revolving unit 1A and the lower traveling unit 1B.
Posture information display image 30 of upper revolving unit 1A and lower traveling unit 1
Although both the posture information display image 40 of B is displayed, only the posture information display image 50 indicating the rotation status and the posture information display image 40 for grasping the inclination of the lower traveling body 1B are displayed during traveling. Alternatively, an inclination sensor may be provided on only one of the upper revolving unit 1A and the lower traveling unit 1B, and only one posture information display image is included in the captured image 15 displayed on the display unit. May be displayed. In the case of a heavy machine that is not divided into the upper revolving unit 1A and the lower traveling unit 1B, for example, a heavy machine such as a bulldozer or a dump truck, only one tilt sensor and one CCD camera may be provided, and the posture based on the signal from the tilt sensor. What is necessary is just to display an information display image.

In the above description, the inclination sensor data and CC
The data is transmitted by synthesizing with the D image signal.
They may be transmitted separately. However, when the transmission is performed separately, such as when a plurality of heavy machines are working together, the number of channels becomes insufficient. Therefore, it is preferable to combine and transmit as described above so that the number of channels is not insufficient.

Embodiment 2 FIG. Also, up to what degree the inclination is within the safe range depends on the heavy equipment. Therefore,
The thresholds of the pitch angle and the roll angle are set in advance according to the heavy equipment to be used, and when the thresholds are exceeded, it is determined that the inclination of the heavy equipment is out of the safe range, and the speaker 22 shown in FIG.
An alarm judgment processing means for executing an alarm process by voice to output an alarm by voice is provided. The alarm determination processing means may be constituted by, for example, a personal computer, a determination program for causing the personal computer to perform the determination, a processing program for causing the alarm processing to be performed, and the like. This processing program
If the determination program determines that the threshold value has been exceeded, the visual information such as changing the display color or form (for example, line type) of the posture information display images 30 and 40, or the color or brightness of the operation TV screen 5f is displayed. May be used as a program for executing the alarm process. Further, the processing program may be a program that executes both the above-mentioned alarm processing by voice and the alarm processing by visual recognition.

The above threshold value is set at an angle that is sufficiently safer than the actual danger angle, that is, with a margin.

According to the second embodiment, if the inclination angle (pitch angle, roll angle) of the heavy equipment exceeds the threshold value, an alarm is automatically issued, so that the remote operator can operate with confidence. And the inclination of the heavy equipment can be corrected earlier, so that the heavy equipment does not fall down.

Embodiment 3 FIG. In the first and second embodiments, the attitude recognition device for a remotely operated heavy machine has been described. The tilt sensor is attached to the heavy machine, and a signal from the tilt sensor is displayed on display means provided in the operation room of the heavy machine. If the posture information of the heavy equipment is displayed based on the above, it is possible to configure a posture recognition device of the heavy equipment in a case of the normal operation performed by the operator getting on the driver's seat of the heavy equipment.

According to the third embodiment, the same effects as those of the first and second embodiments can be obtained in the case of normal operation of heavy equipment.

[0028]

According to the apparatus for recognizing a posture of a heavy equipment according to the present invention, when the heavy equipment is remotely controlled or when the heavy equipment is normally operated, it is possible to easily grasp the front, rear, left and right inclinations of the heavy equipment. Therefore, not only beginners but also experts can easily determine whether the heavy equipment is in a safe posture. Also, by simultaneously displaying the posture information and the photographing information on the screen of one display unit, the user can concentrate on remote control without moving the line of sight extremely.
You will be able to check whether the heavy equipment is in a safe position. Further, in the heavy machinery separated into the upper revolving unit and the lower traveling unit, a sensor is attached to each of the upper revolving unit and the lower traveling unit, and each posture information based on signals from these sensors is displayed on the display means. Therefore, it is possible to easily determine whether the heavy machine that separates into the upper revolving unit and the lower traveling unit has a safe posture, and in this case, it is also possible to grasp the rotation state of the upper revolving unit with respect to the lower traveling unit. It becomes possible. In addition, an alarm determination processing means is provided, and when it is determined that the inclination of the heavy equipment has deviated from the safe range, an alarm is automatically issued, so that the steering can be performed with confidence.

[Brief description of the drawings]

FIG. 1 is a schematic diagram illustrating a posture recognition device for a heavy equipment according to a first embodiment of the present invention.

FIG. 2 is a diagram illustrating a configuration of a heavy equipment side.

FIG. 3 is a diagram illustrating an example of a display image displayed on a display unit in a moving operation room.

FIG. 4 is a block diagram for explaining processing functions and processing procedures on the heavy equipment side and the moving operation room side.

[Explanation of symbols]

1 hydraulic excavator (heavy machine), 3A, 3B tilt sensor (sensor), 4A, 4B CCD camera (photographing means),
5 display means, 5f operation TV screen, 15 photographed image (photographing information), 20 moving operation room, 30, 40 posture information display image (posture information).

Claims (5)

[Claims] 1. A sensor for detecting the inclination of the heavy machine in the front-rear and left-right directions with respect to the horizontal state, and a photographing means for photographing a working condition of the heavy machine are attached to the heavy machine, and provided in a moving operation room for remotely controlling the heavy machine. An apparatus for recognizing a posture of a heavy equipment, wherein the display means displays the posture information of the heavy equipment based on the signal from the sensor and the photographing information from the photographing means. 2. The apparatus according to claim 1, wherein the attitude information and the photographing information are simultaneously displayed on a screen of one display unit. 3. A sensor for detecting inclination of the heavy machine in front, rear, left and right with respect to a horizontal state is attached to the heavy machine, and display means provided in the operation room of the heavy machine displays posture information of the heavy machine based on a signal from the sensor. And a posture recognition device for heavy equipment. 4. In a heavy machinery separated into an upper revolving unit and a lower traveling unit, the sensors are attached to each of the upper revolving unit and the lower traveling unit, and respective attitude information based on signals from these sensors is displayed on the display means. The posture recognition device for a heavy equipment according to any one of claims 1 to 3, wherein: 5. An alarm process by voice or a visual alarm process such as changing the color or form of display of the attitude information when it is determined from the signal from the sensor that the inclination of the heavy equipment is out of the safe range. The attitude recognition device for a heavy equipment according to any one of claims 1 to 4, further comprising an alarm determination processing unit that performs the following.