JP2009227076A - Working vehicle - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a working vehicle having an upper revolving body set on a lower travelling body, which provides an operator with an image in the proceeding direction when it proceeds toward the back side of the operator, regardless of the direction of the upper part revolving body. <P>SOLUTION: This working vehicle is furnished with: a plurality of cameras 50a, 50b, 50c mounted on the upper part revolving body 20 and to photograph directions different from each other; an image signal processing part 3513 to extract spectrum information to show movement of an object in the image which the same camera photographs for each of the cameras in accordance with the images which the cameras 50a, 50b, 50c respectively photograph when a power shovel 1 travels by the lower part travelling body 10; a control processing part 3515 to select one of the cameras in accordance with the extracted spectrum information for each of the cameras; and a monitor 340 to display the image which the selected camera photographs. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  TECHNICAL FIELD The present invention relates to a work vehicle in which an upper turning body is installed on a lower traveling body, the surrounding situation is photographed with a camera, and displayed on a display screen that is referred to by an operator. It is related with the technology to display.

  In a work vehicle such as a power shovel in which an upper swinging body is installed on the lower traveling body, cameras are installed on both sides of the rear part of the upper swinging body, and based on information from the operation lever when the upper swinging body turns. A technique for selecting a camera in a turning direction and displaying an image captured by the selected camera on a monitor is known (for example, Patent Document 1).

Further, in a work vehicle, when the work vehicle moves backward, that is, when the work vehicle travels in the direction behind the operator, it is generally performed that the traveling direction is photographed with a camera and the image is provided to the operator. Yes.
JP 2001-296321 A

  However, in a work vehicle in which the upper turning body can turn with respect to the lower traveling body, the work vehicle can travel regardless of the orientation of the upper turning body with respect to the lower traveling body. Accordingly, when the camera is mounted on the upper swing body, the shooting direction of the camera also rotates with the upper swing body, and thus the traveling direction cannot always be captured.

  Accordingly, an object of the present invention is to provide an image of the traveling direction of a work vehicle in which an upper turning body is installed on a lower traveling body when the vehicle travels behind the operator regardless of the orientation of the upper turning body. Is provided to the operator.

  A work vehicle according to one embodiment of the present invention includes a lower traveling body (10) and an upper turning body (20) installed on an upper portion of the lower traveling body (10). Then, the work vehicle (1) travels by a plurality of photographing means (50a, 50b, 50c) attached to the upper swing body (20) for photographing different directions and the lower traveling body (10). Based on the images photographed by the plurality of photographing means (50a, 50b, 50c), the vector information indicating the movement of the object in the images photographed by the respective photographing means is obtained. (50a, 50b, 50c) Image processing means (3513) to be extracted separately, and selection to select one photographing means based on the vector information for each of the plurality of photographing means extracted by the image processing means (3513) Means (3515) and display means (340) for displaying an image photographed by the photographing means selected by the selection means (3515).

  In a preferred embodiment, the plurality of photographing means (50a, 50b, 50c) may photograph different directions behind the operator (OP) who operates the work vehicle (1). Good.

  In a preferred embodiment, the selection means (3515) estimates the traveling direction of the work vehicle (1) based on the vector information for each of the plurality of photographing means, and the work vehicle whose photographing direction is estimated. You may make it select the one imaging | photography means which has faced the advancing direction of (1).

  In a preferred embodiment, the selection means (3515) includes each of the plurality of photographing means (50a, 50b, 50c) from vector information indicating the movement of an object in an image photographed by the same photographing means. A vector component in a predetermined direction in the image may be extracted, and one image capturing unit that captures an image having the largest extracted vector component may be selected.

  In a preferred embodiment, the image processing means (3513) extracts, as the vector information, an optical flow in an image photographed by the same photographing means for each of the plurality of photographing means (50a, 50b, 50c). May be.

  Hereinafter, a power shovel will be described as an example of a work vehicle according to an embodiment of the present invention with reference to the drawings.

  FIG. 1 is a side view of a power shovel 1 according to the present embodiment, and FIG. 2 is a plan view schematically showing the power shovel 1 according to the present embodiment.

  As shown in FIG. 1, the excavator 1 is provided on a crawler-type lower traveling body 10, an upper revolving body 20 that is pivotably provided on an upper portion of the lower traveling body 10, and an upper portion of the upper revolving body 20. A cab 30 and a work implement 40 provided on the front side of the upper swing body 20 (on the front side when the operator OP is seated in the cab 30 and on the left side in FIG. 1).

  As shown in FIGS. 1 and 2, a plurality of cameras 50 (50 a, 50 b, 50 c) are attached to the upper swing body 20. The cameras 50a, 50b, and 50c are attached in different directions. For example, they are attached in different directions on the rear side of the upper swing body 20 (the back side of the operator OP seated in the cab 30). In the example of FIG. 2, the camera 50a is directed diagonally to the left of the operator OP, the camera 50b is directed directly behind the operator OP, and the camera 50c is directed to the diagonally rear right of the operator OP.

  Each camera 50a, 50b, 50c captures the direction in which it is directed. Since each of the cameras 50a, 50b, and 50c is fixed to the upper swing body 20, the direction in which each of the cameras 50a, 50d, and 50c is changed (the direction in which the camera 50a, 50b, and 50c is directed) changes.

  Further, in the power shovel 1, the upper swing body 20 freely rotates with respect to the lower traveling body 10, and therefore, as shown in FIG. 2A, the direction in which the operator OP is facing (front) and the direction of the lower traveling body 10 are There are cases where they match and cases where they do not match as shown in FIG. 2B.

  In addition, the power shovel 1 according to the present embodiment is equipped with a video display system that provides a video taken by the camera 50 to the operator OP on the monitor 340 in the cab 30. FIG. 3 shows the configuration of this video display system.

  As shown in the figure, this video display system includes a plurality of cameras 50a, 50b, 50c, a monitor 340 installed in the cab 30, and videos taken by the cameras 50a, 50b, 50c. An image processing device 350 that outputs a horizontally reversed image (mirror image) of any one of the images selected from the above to a monitor 340, and a cable such as a camera harness that connects the camera 50 and the image processing device 350.

  Here, the image processing apparatus 350 includes an operation unit 352 including various keys for accepting the operation of the operator OP, and an image processing unit 351 that performs various processes instructed by signals from the operation unit 352. The image processing unit 351 performs various image processing such as left-right reversal and vector calculation on the captured images of the input control unit 3511 and the cameras 50a, 50b, and 50c to which the output signals of the cameras 50a, 50b, and 50c are sequentially input. A video signal processing unit 3513 to be applied, an image memory unit 3514 for storing a frame image, a control processing unit 3515 for selecting any one of the images captured by the cameras 50a, 50b, and 50c, and an image selected by the control processing unit 3515 An output control unit 3516 for displaying the horizontally reversed video on the monitor 340 is provided. Such an image processing unit 351 is realized by, for example, an image processing custom IC, a CPU, a memory, and the like.

  The image memory unit 3514 stores video data captured by each camera, that is, time-series frame image data.

  The video signal processing unit 3513 converts the video signal input from the input control unit 3511 into image data for each frame, and sequentially stores it in the image memory unit 3514. The video signal processing unit 3513 performs predetermined image processing on the image data stored in the image memory unit 3514. For example, the video signal processing unit 3513 calculates, for each camera 50, vector information of an object shown in a plurality of images taken at different times. For example, the video signal processing unit 3513 calculates an optical flow for each camera 50 using a plurality of frame images taken by the same camera.

  The control processing unit 3515 selects one camera 50 based on the vector information for each camera 50 calculated by the video signal processing unit 3513. For example, the control processing unit 3515 estimates the traveling direction of the power shovel 1 based on the movement vector for each camera 50 calculated by the video signal processing unit 3513, and selects the camera 50 capturing the traveling direction.

  Next, the backward operation support process executed in such a video display system will be described.

  FIG. 4 shows an overall flowchart of processing of the video display system according to the present embodiment.

  When the operator OP operates the power switch of the operation unit 352, the camera power is turned on (S100). Thereby, each camera 50 starts photographing. In this initial state, the camera 50b is selected, and an image captured by the camera 50b is displayed on the monitor 340.

  When the video display system is operating, the power shovel 1 operates according to the operation of the operator OP. For example, the upper turning body 20 turns or the work implement 40 operates, and the power shovel 1 travels by the lower traveling body 10 (S200). In the present embodiment, the following processing is performed particularly on an image captured when the excavator 1 travels.

  That is, the video signal processing unit 3513 calculates an optical flow for each camera 50 based on the frame image of each camera 50 taken while the excavator 1 is traveling (S300). Further, the control processing unit 3515 selects the camera 50 facing the traveling direction of the power shovel 1 based on the optical flow for each camera 50 calculated by the video signal processing unit 3513 (S400). Then, the control processing unit 3515 instructs the video signal processing unit 3513 to display the video captured by the camera selected here on the monitor 340.

  In accordance with an instruction from the control processing unit 3515, the video signal processing unit 3513 outputs the video captured by the camera selected in step S400 to the output control unit 3516 and displays it on the monitor 340 in real time (S500).

  As a result, when the excavator 1 travels to the back side of the operator OP, the camera 50 that automatically faces in the traveling direction is automatically captured, but the image can be displayed on the monitor 340. As a result, the operator can grasp the state of the traveling direction in real time.

  Next, FIG. 5 is a flowchart showing a detailed procedure of the optical flow calculation process in step S300. FIG. 6 is an explanatory diagram of the optical flow calculation process. Hereinafter, a description will be given with reference to FIGS.

  First, the video signal processing unit 3513 obtains image data of the central band-like region 520a portion of the frame image 510a at time t from the image memory unit 3514 (S310). Then, a predetermined filter is applied to the obtained image data of the band-like region 520a to generate a filtered image T1 (S320). Then, after waiting for the time Δt to elapse, the next process is performed (S330).

  The video signal processing unit 3513 acquires the image data of the band-like region 520b at the center of the frame image 510b at the time (t + Δt) from the image memory unit 3514 (S340). Then, a filtered image T2 is generated by applying a predetermined filter to the acquired image data of the band-like region 520b (S350).

  Then, the video signal processing unit 3513 calculates an optical flow image 530 based on the density gradient of the central strip regions 520a and 520b from the filter processed images T1 and T2 (S360).

  Thus, the optical flow calculated by the video signal processing unit 3513 is only the central band-like region 520, and can be processed at high speed. In addition, in a video shot with a camera that faces straight in the traveling direction, the vertical vector near the center increases. Therefore, this is used in the next camera selection process.

  FIG. 7 is a flowchart showing a detailed procedure of the camera selection process in step S400. FIG. 8 is an explanatory diagram of this camera selection process. Hereinafter, a description will be given with reference to FIGS. In the initial state such as immediately after the power shovel 1 is activated, the camera 50b facing directly behind the operator OP is selected.

  First, the control processing unit 3515 decomposes each optical flow into an X direction (horizontal direction) component and a Y direction (vertical direction) component based on the optical flow image 535 of each camera calculated in step S300 (S410). . For example, referring to FIG. 8, an optical flow image 535a obtained from an image taken by the camera 50a, an optical flow image 535b obtained from an image taken by the camera 50b, and an optical flow image 535c obtained from an image taken by the camera 50c. Then, the control processing unit 3515 decomposes the optical flow vector into an X direction component and a Y direction component. Since the optical flow image 535 is generated from the band-like region 520 (see FIG. 6) at the center of the frame image 510, the X direction and Y direction of the optical flow image 535 are the X direction and Y of the frame image 510. The direction is the same.

  The control processing unit 3515 compares the Y-direction components of the optical flow of each camera disassembled in step S410 to obtain the maximum value (S420). For example, in the example of FIG. 8, the maximum values of the Y direction components of the optical flows of the optical flow images 535a to 535c are respectively compared to obtain the largest value.

  The control processing unit 3515 determines whether or not the maximum value of the Y direction component of the optical flow acquired in step S420 is equal to or greater than a predetermined threshold value (S430).

  If the maximum value of the Y direction component of the optical flow is equal to or greater than a predetermined threshold value in step S430 (S430: Yes), the optical flow image 535 having the maximum value of the Y direction component of the optical flow is captured. The camera 50 is selected. On the other hand, if the maximum value of the Y direction component of the optical flow is smaller than a predetermined threshold value in step S430 (S430: No), no camera is selected and the currently selected camera is maintained as it is. To do.

For example, in the example of FIG. 8, when the maximum value OF _max of the Y direction component of the optical flow image 535c is larger than the maximum value of the Y direction component of the optical flow images 535a and 535b and larger than a predetermined threshold value, The camera 50c is selected.

  Thus, by selecting a camera that has captured a frame image having the largest vector in the Y direction from among a plurality of cameras 50, an image captured by the camera 50 that is most facing in the traveling direction is automatically monitored. 340 is displayed. As a result, the operator OP can surely confirm the situation in the traveling direction without performing the camera switching operation himself.

  The above-described embodiments of the present invention are examples for explaining the present invention, and are not intended to limit the scope of the present invention only to those embodiments. Those skilled in the art can implement the present invention in various other modes without departing from the gist of the present invention.

  For example, the optical flow is calculated as the vector information in the above-described embodiment, but the vector information may be calculated by a method other than this. For example, a vector indicating the motion of the object in the frame image may be calculated by an inter-frame difference method or the like.

  In the above-described embodiment, three cameras are used. However, any number of cameras may be used as long as the number of cameras is two or more.

It is a side view of the power shovel 1 concerning this embodiment. It is the top view which showed typically the power shovel 1 which concerns on this embodiment. 1 is a configuration diagram of a video display system mounted on a power shovel 1 according to the present embodiment. It is a whole flowchart of a process of the video display system concerning this embodiment. It is a flowchart which shows the detailed procedure of the optical flow calculation process of step S300. It is explanatory drawing of the optical flow calculation process of step S300. It is a flowchart which shows the detailed procedure of the camera selection process of step S400. It is explanatory drawing of the camera selection process of step S400.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Power shovel, 10 ... Lower traveling body, 20 ... Upper turning body, 30 ... Driver's cab, 40 ... Working machine, 50a, 50b, 50c ... Camera, 340 ... Monitor, 350 ... Image processing apparatus, 351 ... Image processing part 352: Operation unit.

Claims (5)

A work vehicle (1) having a lower traveling body (10) and an upper turning body (20) installed on an upper portion of the lower traveling body (10),
A plurality of photographing means (50a, 50b, 50c) attached to the upper swing body (20) for photographing different directions;
Based on images taken by the plurality of photographing means (50a, 50b, 50c) when the work vehicle (1) is traveling by the lower traveling body (10), images taken by the respective photographing means Image processing means (3513) for extracting vector information indicating the movement of the object in the plurality of photographing means (50a, 50b, 50c),
A selection unit (3515) for selecting one imaging unit based on the vector information for each of the plurality of imaging units extracted by the image processing unit (3513);
A work vehicle comprising: display means (340) for displaying an image photographed by the photographing means selected by the selection means (3515).   The work vehicle according to claim 1, wherein the plurality of photographing means (50a, 50b, 50c) photograph different directions behind an operator (OP) who operates the work vehicle (1).   The selection means (3515) estimates the traveling direction of the work vehicle (1) based on the vector information for each of the plurality of photographing means, and the traveling direction of the work vehicle (1) whose photographing direction is estimated. The work vehicle according to claim 1, wherein one photographing means facing the vehicle is selected.   The selection means (3515) is determined in advance in each image from vector information indicating the movement of an object in an image taken by the same photographing means for each of the plurality of photographing means (50a, 50b, 50c). The work vehicle according to any one of claims 1 to 3, wherein a vector component in the selected direction is extracted, and one image capturing unit that captures an image having the largest extracted vector component is selected.   The image processing means (3513) extracts, as the vector information, an optical flow in an image photographed by the same photographing means for each of the plurality of photographing means (50a, 50b, 50c). A work vehicle according to any one of the above.

Images