US20130155240A1 - Work vehicle periphery monitoring apparatus - Google Patents

Work vehicle periphery monitoring apparatus Download PDF

Info

Publication number
US20130155240A1
US20130155240A1 US13/818,323 US201213818323A US2013155240A1 US 20130155240 A1 US20130155240 A1 US 20130155240A1 US 201213818323 A US201213818323 A US 201213818323A US 2013155240 A1 US2013155240 A1 US 2013155240A1
Authority
US
United States
Prior art keywords
vehicle body
camera
display
work vehicle
vehicle
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Abandoned
Application number
US13/818,323
Other languages
English (en)
Inventor
Shinji Mitsuta
Shigeru Harada
Tomikazu Tanuki
Eishin Masutani
Yukihiro Nakanishi
Takeshi Kurihara
Dai Tsubone
Masaomi Machida
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Komatsu Ltd
Original Assignee
Komatsu Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Komatsu Ltd filed Critical Komatsu Ltd
Assigned to KOMATSU LTD reassignment KOMATSU LTD ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: HARADA, SHIGERU, KURIHARA, TAKESHI, MACHIDA, Masaomi, MASUTANI, EISHIN, MITSUTA, SHINJI, NAKANISHI, YUKIHIRO, TANUKI, TOMIKAZU, TSUBONE, Dai
Publication of US20130155240A1 publication Critical patent/US20130155240A1/en
Abandoned legal-status Critical Current

Links

Images

Classifications

    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R1/00—Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
    • B60R1/002—Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles specially adapted for covering the peripheral part of the vehicle, e.g. for viewing tyres, bumpers or the like
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R1/00—Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
    • B60R1/20—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
    • B60R1/22—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles for viewing an area outside the vehicle, e.g. the exterior of the vehicle
    • B60R1/23—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles for viewing an area outside the vehicle, e.g. the exterior of the vehicle with a predetermined field of view
    • B60R1/26—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles for viewing an area outside the vehicle, e.g. the exterior of the vehicle with a predetermined field of view to the rear of the vehicle
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R1/00—Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
    • B60R1/20—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
    • B60R1/22—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles for viewing an area outside the vehicle, e.g. the exterior of the vehicle
    • B60R1/23—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles for viewing an area outside the vehicle, e.g. the exterior of the vehicle with a predetermined field of view
    • B60R1/27—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles for viewing an area outside the vehicle, e.g. the exterior of the vehicle with a predetermined field of view providing all-round vision, e.g. using omnidirectional cameras
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/63—Control of cameras or camera modules by using electronic viewfinders
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/698—Control of cameras or camera modules for achieving an enlarged field of view, e.g. panoramic image capture
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
    • H04N7/181—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast for receiving images from a plurality of remote sources
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
    • B60R2300/10—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of camera system used
    • B60R2300/105—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of camera system used using multiple cameras
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
    • B60R2300/30—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of image processing
    • B60R2300/304—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of image processing using merged images, e.g. merging camera image with stored images
    • B60R2300/305—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of image processing using merged images, e.g. merging camera image with stored images merging camera image with lines or icons
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
    • B60R2300/60—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by monitoring and displaying vehicle exterior scenes from a transformed perspective
    • B60R2300/602—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by monitoring and displaying vehicle exterior scenes from a transformed perspective with an adjustable viewpoint
    • B60R2300/605—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by monitoring and displaying vehicle exterior scenes from a transformed perspective with an adjustable viewpoint the adjustment being automatic
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
    • B60R2300/60—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by monitoring and displaying vehicle exterior scenes from a transformed perspective
    • B60R2300/607—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by monitoring and displaying vehicle exterior scenes from a transformed perspective from a bird's eye viewpoint
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
    • B60R2300/80—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the intended use of the viewing arrangement
    • B60R2300/8093—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the intended use of the viewing arrangement for obstacle warning
    • E—FIXED CONSTRUCTIONS
    • E02—HYDRAULIC ENGINEERING; FOUNDATIONS; SOIL SHIFTING
    • E02F—DREDGING; SOIL-SHIFTING
    • E02F9/00—Component parts of dredgers or soil-shifting machines, not restricted to one of the kinds covered by groups E02F3/00 - E02F7/00
    • E02F9/26—Indicating devices
    • E02F9/261—Surveying the work-site to be treated

Definitions

  • the present invention relates to a periphery monitoring apparatus, and in particular, to a periphery monitoring apparatus which displays a camera image of the surroundings of a work vehicle on a display apparatus.
  • dump trucks which are used for carrying crushed rock at a mine or the like are known.
  • the periphery monitoring apparatus of non patent literature 1 is provided with three cameras which video the front, left, and right of the work vehicle and one rear camera which videos the rear of the work vehicle.
  • the rear camera is disposed below the vehicle body which has a vessel and the ground surface at the rear of the work vehicle appears in the camera image.
  • the present invention has been carried out in consideration of the circumstances described above and has an object of providing a periphery monitoring apparatus where it is possible to easily grasp the positional relationship of the vehicle body and objects in the periphery of the vehicle body using a camera image.
  • a periphery monitoring apparatus for a work vehicle of a first aspect displays an image of the surroundings of the work vehicle on a display apparatus
  • the periphery monitoring apparatus includes a camera being disposed below a vehicle body of the work vehicle, the camera being configured to acquire a camera image of the surroundings of the work vehicle; and a display control section being configured to display the camera image acquired using the camera and a vehicle body outer edge line on the display apparatus, the vehicle body outer edge line indicating an outer edges of the vehicle body which is vertically projected on a ground surface in the camera image.
  • the periphery monitoring apparatus for a work vehicle of the first aspect it is possible for the driver to easily grasp the positional relationship of the vehicle body which is positioned above the camera and objects in the periphery of the vehicle body (for example, a person, a vehicle, a building, or the like) with the vehicle body outer edge line as a reference.
  • the camera image for example, if a person has entered the space below the vehicle body, to what extent that the vehicle body has approached a building, or the like.
  • a periphery monitoring apparatus for a work vehicle of a second aspect is related to the first aspect, the display control section is configured to display a warning line on an outside of the vehicle body outer edge line, the warning line indicating a position with a predetermined distance from the vehicle body outer edge line on the ground surface in the camera image.
  • the periphery monitoring apparatus for a work vehicle of the second aspect it is possible for the driver to more easily grasp the positional relationship of the vehicle body and objects in the periphery of the vehicle body since to what extent objects in the periphery of the vehicle body are separated from the vehicle body is shown along with the vehicle body outer edge line in a stepped manner.
  • a periphery monitoring apparatus for a work vehicle of a third aspect is related to the first or the second aspect, the vehicle body has a frame and a vessel disposed on the frame, a rear edge portion of the vessel is included in an imaging range of the camera, and the vehicle body outer edge line shows the rear edge portion of the vessel which is vertically projected on the ground surface in the camera image.
  • the periphery monitoring apparatus for a work vehicle of the third aspect it is possible for the driver to more clearly grasp the positional relationship of the vessel and objects in the periphery of the vessel since it is possible for the rear edge portion of the vessel to be directly confirmed in the camera image.
  • a periphery monitoring apparatus for a work vehicle of a fourth aspect displays an image of the surroundings of the work vehicle on a display apparatus and includes a plurality of cameras being mounted on a vehicle body of the work vehicle, each of the plurality of cameras being configured to acquire a camera image of the surroundings of the work vehicle; and a display control section being configured to lines up a bird's-eye image of the surroundings of the work vehicle and a specific camera image out of a plurality of camera images acquired using the plurality of cameras on the camera image, the bird's-eye image being acquired based on the plurality of images, the display control section being configured to display a vehicle body outer edge line in the specific camera image, the vehicle body outer edge line indicating an outer edges of the vehicle body which is vertically projected on a ground surface in the specific camera image.
  • the present invention it is possible to provide a periphery monitoring apparatus where it is possible to easily determine the positional relationship of the vehicle body and objects in the periphery of the vehicle body using a camera image.
  • FIG. 1 is a perspective diagram illustrating an overall configuration of a dump truck.
  • FIG. 2 is a block diagram illustrating a configuration of a periphery monitoring apparatus.
  • FIG. 3 is a planar diagram of a dump truck which illustrates mounting positions of six cameras and two-dimensional videoing ranges.
  • FIG. 4 is a perspective diagram of a dump truck which illustrates three-dimensional imaging ranges using six cameras.
  • FIG. 5 is a perspective diagram illustrating an attachment state of six cameras.
  • FIG. 6 is a schematic diagram illustrating a configuration of a monitor.
  • FIG. 7 is a schematic diagram illustrating a configuration of a monitor.
  • front, back, left, and right are terms with an operator who is sitting in a driver's seat as a reference, and “vehicle width direction” has the same meaning as “left and right direction”.
  • FIG. 1 is a perspective diagram illustrating an overall configuration of a dump truck 1 which is provided with a periphery monitoring apparatus according to an embodiment of the present invention.
  • the dump truck 1 is a self-propulsion type of extra-large work vehicle which is used in mining operations and the like.
  • the dump truck 1 is mainly provided with a vehicle body frame 2 , a cab 3 , a vessel 4 , two front wheels 5 , two rear wheels 6 , and a base 7 .
  • the dump truck 1 is provided with a periphery monitoring apparatus 10 which displays an image of the surroundings of the vehicle on a monitor. The configuration of the periphery monitoring apparatus 10 will be described later.
  • the vehicle body frame 2 supports a diesel engine, a driving mechanism such as a transmission unit, other auxiliary units, and the like which are not shown.
  • the front wheels 5 (only the right front wheel is shown in the diagram in FIG. 1 ) are supported in the left and right of the front portion of the vehicle body frame 2
  • the rear wheels 6 (only the right rear wheel is shown in the diagram in FIG. 1 ) are supported in the left and right of the rear portion of the vehicle body frame 2 .
  • the vehicle body frame 2 has a lower deck 2 A at a side which is close to the ground surface and has an upper deck 2 B above the lower deck 2 A.
  • Two movable ladders 2 C are provided between the lower deck 2 A and the ground surface and a sloping ladder 2 D is provided between the lower deck 2 A and the upper deck 2 B.
  • Hand rails with a palisade shape are fixed in the left and right of the front portion, the side portion, and a portion of the back portion on the upper deck 2 B.
  • the cab 3 is disposed to be lateralized towards the left side from the center in the vehicle width direction above the upper deck 2 B.
  • a driver's seat, a shift lever, a controller, a monitor, a handle, an acceleration pedal, a brake pedal, and the like are provided in the cab 3 .
  • the controller, the monitor, and the shift level configure a portion of the periphery monitoring apparatus 10 .
  • the vessel 4 is a container for hauling heavy materials such as crushed rocks.
  • a bottom portion at the rear side of the vessel 4 is connected to a rear edge portion of the vehicle body frame 2 so as to be able to rotate via a rotation pin (which is not shown). Due to this, it is possible for the vessel 4 to take an erect posture, where loading materials are discharged by the front portion being rotated upward using an actuator such as a hydraulic cylinder which is not shown, and a hauling posture where the front portion is position at an upper portion of the cab as shown in FIG. 1 .
  • the base 7 is disposed above the upper deck 2 B.
  • the base 7 is a power supply apparatus for positioning a pantograph which receives electrical power from a bird's-eye wire such as a trolley.
  • vehicle body frame 2 and the vessel 4 configure the “vehicle body” of the dump truck 1 in the embodiment.
  • FIG. 2 is a block diagram illustrating a configuration of the periphery monitoring apparatus 10 which is provided in the dump truck 1 .
  • the periphery monitoring apparatus 10 has six cameras 11 to 16 , six frame memories 11 A to 16 A, an operator switch 20 , a monitor 30 , and a display control section 40 .
  • FIG. 3 is a planar diagram of the dump truck 1 which illustrates mounting positions of the six cameras 11 to 16 and two-dimensional videoing ranges.
  • FIG. 4 is a perspective diagram of the dump truck 1 which illustrates three-dimensional imaging ranges using the six cameras 11 to 16 .
  • FIG. 5 is a perspective diagram illustrating an attachment state of the six cameras 11 to 16 .
  • the six cameras 11 to 16 are mounted on the outside of the dump truck 1 and acquire images of the surroundings of the dump truck 1 .
  • a first camera 11 is disposed at an upper edge portion of the sloping ladder 2 D and is fixed toward the front of the vehicle.
  • a first videoing range 11 R of the first camera 11 is the front of the vehicle.
  • a second camera 12 is disposed at a right edge portion on a front side surface of the upper deck 2 B and is fixed toward the front right diagonal of the vehicle.
  • a second videoing range 12 R of the second camera 12 is the front right diagonal of the vehicle.
  • a third camera 13 is disposed at a position which is left and right symmetrical with the second camera 12 , that is, a left edge portion on a front side surface of the upper 2 B and is fixed toward the front left diagonal of the vehicle.
  • a third videoing range 13 R of the third camera 13 is the front left diagonal.
  • a fourth camera 14 is disposed at a front edge portion on a right side surface of the upper deck 2 B and is fixed toward the rear right diagonal of the vehicle.
  • a fourth videoing range 14 R of the fourth camera 14 is the rear right diagonal.
  • a fifth camera 15 is disposed at a position which is left and right symmetrical with the fourth camera 14 , that is, a front edge portion on a left side of the upper deck 2 B and is fixed toward the rear left diagonal of the vehicle.
  • a fifth videoing range 15 R of the fifth camera 15 is the rear left diagonal.
  • a sixth camera 16 is disposed to be beneath the vessel 4 as shown in FIG. 5 and is fixed toward the rear of the vehicle. Specifically, the sixth camera 16 is fixed by a bracket 9 a which is attached to a cross bar 9 above a rear axel 8 . The sixth camera 16 is fixed diagonally downward toward the rear and a sixth imaging range 16 R of the sixth camera 16 is the rear of the vehicle below the vessel 4 .
  • the vehicle body itself (the rear edge portion of the vessel 4 ) is included in the sixth imaging range 16 R of the sixth camera 16 (refer to FIG. 6 ) which is different to the imaging range images of the other cameras.
  • Images of substantially the entire surroundings of the dump truck 1 are acquired using the six cameras 11 to 16 above.
  • Each of the six cameras 11 to 16 respectively transmit camera image data, which indicates the camera image which is videoed by the apparatus itself, to the frame memories 11 A to 16 A.
  • the six frame memories 11 A to 16 A are connected to the six cameras 11 to 16 .
  • the six frame memories 11 A to 16 A temporarily store the camera image data which is received from the six cameras 11 to 16 .
  • the six frame memories 11 A to 16 A transmit the camera image data which has been stored to the display control section 40 .
  • the operator switch 20 is an operating member for specifying the camera image, which is to be displayed on the monitor 30 , out of the camera images from each of the six cameras 11 to 16 . It is possible for a desired camera image to the displayed on the monitor 30 by the driver operating the operator switch 20 . The operator switch 20 notifies the display control section 40 whether any of the camera images has been selected.
  • FIG. 6 and FIG. 7 are schematic diagrams illustrating a configuration of the monitor 30 .
  • the monitor 30 is disposed in front of the driver's seat in the cab 3 .
  • the monitor 30 has a display region 30 a which displays the camera image as shown in FIG. 6 and FIG. 7 .
  • FIG. 6 shows an appearance where a camera image 16 G of the sixth camera 16 and a bird's-eye image F are displayed in the display region 30 a.
  • FIG. 7 shows an appearance where a camera image 15 G of the fifth camera 15 and the bird's-eye image F are displayed in the display region 30 a.
  • a plurality of camera images may be displayed in the display region 30 a or only the bird's-eye image F may be displayed in the display region 30 a.
  • the display of the monitor 30 may be automatically switched according to the movement state of the dump truck 1 .
  • the camera image of the first camera and the bird's-eye image F may be displayed during forward progression and when stationary and only the camera image 16 G of the sixth camera 16 to be displayed during backward progression.
  • the camera image 16 G and the bird's-eye image F may be displayed during backward progression and the camera image 16 G and another camera image may be displayed during backward progression.
  • the display control section 40 selects camera image data which corresponds to the camera image which has been selected based on the notification from the operator switch 20 .
  • the display control section 40 generates bird's-eye image data where the plurality of images are videoed from a predetermined projected plane due to conversion of the coordinates of the camera image data of the six cameras 11 to 16 and which shows the bird's-eye image F which encompasses the entire surroundings of the dump truck 1 .
  • the display control section 40 generates two-screen data which corresponds to a two-screen image where the camera image which has been selected and the bird's-eye image F are lined up due to the combination of the camera image data and the bird's-eye image data.
  • the display control section 40 generates vehicle body outer edge line data which corresponds to a vehicle body outer edge line BL showing the outer edges of the vehicle body (including the vehicle body frame 2 and the vessel 4 ) which is vertically projected with regard to the ground surface in the camera image which has been selected.
  • the display control section 40 generates the vehicle body outer edge line data according to distortions of the camera image of a target camera in accordance with a sequence which will be described next.
  • the display control section 40 defines three vectors Ra, Rb, and Rc which indicate the orientation of the target cameras based on the following formulae (1) to (3).
  • the display control section 40 converts the three-dimensional coordinates (Xr, Yr, Zr) at one point on the outer edge of the vehicle body, which is vertically projected with regard to the ground surface, in a physical coordinates system to three-dimensional coordinates (x, y, z) in a camera coordinates system where the orientation and the position of the target camera is a reference as an Euler angle in the vertical direction—front and back direction—vehicle width direction based on the following formula (4).
  • (tx, ty, tz) are coordinates in the physical coordinates system of the target camera.
  • the display control section 40 converts the three-dimensional coordinates (x, y, z) in the camera coordinates system to two-dimensional coordinates (u, v) in the display region 30 a of the monitor 30 based on the following formulae (5) to (11).
  • x 3 x 2(1+ k 1 *r 2 +k 2* r 4 +k 3* r 6 )+2* p 1 *x 2* y 2+ p 2( r 2 +2* x 2 2 ) (8)
  • y 3 y 2(1+ k 1 *r 2 +k 2* r 4 +k 3* r 6 )+ p 1( r 2 +2* y 2 2 ) +2* p 2* x 2* y 2 (9)
  • k1, k2, and k3 are radial direction strain coefficients
  • P 1 and p 2 are shear direction strain coefficients
  • fx and fy are camera focal distances
  • cx and cy are camera coordinate systems with the focal direction of the camera as the Z axis, the imaging plane direction as the X axis and the Y axis.
  • the display control section 40 acquires the two-dimensional coordinates (u, v) at one point on the outer edge of the vehicle, which is vertically projected with regard to the ground surface, in the display region 30 a.
  • the display control section 40 generates the vehicle body outer edge line data which corresponds to the vehicle body outer edge line BL with a desired form (a dashed line, a chain line, or the like) by acquiring the two-dimensional coordinates in the same manner with regard to a plurality of points on the outer edges of the vehicle body.
  • the display control section 40 generates warning line data which corresponds the a first to a third warning lines L 1 to L 3 which indicate positions with predetermined distances on the outside from the vehicle body outer edge line BL using the formulae (1) to (11) described above in the same manner.
  • the first warning line L 1 is a line which is 3 m to the outside of the vehicle body outer edge line BL
  • the second warning line L 2 is a line which is 5 m to the outside of the vehicle body outer edge line BL
  • the third warning line L 3 is a line which is 7 m to the outside of the vehicle body outer edge line BL.
  • the display control section 40 generates display image data by overlapping the two-screen data which has been generated, the vehicle body outer edge line data which has been generated, and the warning line data which has been generated.
  • the display control section 40 transmits the display image data which has been generated to the monitor 30 . Due to this, the vehicle body outer edge line BL, which indicates the outer edges of the vehicle body which is vertically projected with regard to the ground surface, and the first to the third warning lines L 1 to L 3 , which indicates the lines which are 3 m, 5 m, and 7 m more to the outside of the vehicle body than the vehicle body outer edge line BL, are displayed along with the camera images 15 G and 16 G on the monitor 30 as shown in FIG. 6 and FIG. 7 .
  • the periphery monitoring apparatus 10 is provided with the display control section 40 which displays the camera image and the vehicle body outer edge line BL, which indicates the outer edges of the vehicle body (including the vessel 4 ) which is vertically projected with regard to the ground surface in the camera image, on the monitor 30 .
  • the driver to easily grasp the positional relationship of the vehicle body which is positioned above the camera and an object (for example, a person, a vehicle, a building, or the like) in the periphery of the vehicle body with the vehicle body outer edge line as a reference.
  • an object for example, a person, a vehicle, a building, or the like
  • it is possible to easily determine using the camera image for example, if a person has entered the space below the vehicle body, to what extent that the vehicle body has approached a building, or the like.
  • the display control section 40 displays the first to the third warning lines L 1 to L 3 , which indicate positions with predetermined distances from the vehicle body outer edge line BL, on the monitor 30 .
  • the ground surface is included in the lower part of the screen in the imaging range 16 R of the sixth camera 16 as shown in FIG. 6 , and the lower surface side of the rear edge portion of the vessel 4 which configures a portion of the vehicle body is included in an upper part of the screen which opposes the ground surface.
  • the display control section 40 displays the first to the third warning lines L 1 to L 3 on the monitor 30 but the present invention is not limited to this.
  • the display control section 40 may display any of the first to the third warning lines L 1 to L 3 , and in addition, may display four or more warning lines.
  • the display control section 40 shows the vehicle body outer edge line BL and the first to the third warning lines L 1 to L 3 with dashed lines but the present invention is not limited to this.
  • the display control section 40 may show the vehicle body outer edge line BL and the first to the third warning lines L 1 to L 3 with a solid line, a chain line, or the like.
  • the display control section 40 may show each of the vehicle body outer edge line BL and the first to the third warning lines L 1 to L 3 with different colors.
  • (C) In the embodiment described above, the case is described where the vehicle body outer edge line BL is displayed along with the camera image 16 G of the sixth camera 16 or the camera image 15 G of the fifth camera 15 using FIG. 6 and FIG. 7 but the present invention is not limited to this.
  • the vehicle body outer edge line BL may be displayed along with the camera images of the first to the fourth cameras in a case where the driver specifies any of the first to the fourth cameras 11 to 14 .
  • the dump truck 1 is provided with the base 7 but need not be provided with the base 7 .
  • the dump truck 1 is provided with six cameras 11 to 16 but the present invention is not limited to this.
  • the dump truck 1 may be provided with one to five cameras or may be provided with seven or more cameras.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Mechanical Engineering (AREA)
  • Closed-Circuit Television Systems (AREA)
  • Component Parts Of Construction Machinery (AREA)
  • Fittings On The Vehicle Exterior For Carrying Loads, And Devices For Holding Or Mounting Articles (AREA)
US13/818,323 2011-06-07 2012-05-24 Work vehicle periphery monitoring apparatus Abandoned US20130155240A1 (en)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
JP2011127478A JP5597596B2 (ja) 2011-06-07 2011-06-07 作業車両の周辺監視装置
JPJP2011-127478 2011-06-07
PCT/JP2012/063255 WO2012169362A1 (ja) 2011-06-07 2012-05-24 作業車両の周辺監視装置

Related Parent Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2012/063255 A-371-Of-International WO2012169362A1 (ja) 2011-06-07 2012-05-24 作業車両の周辺監視装置

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US15/392,458 Division US9956915B2 (en) 2011-06-07 2016-12-28 Dump truck periphery monitoring apparatus

Publications (1)

Publication Number Publication Date
US20130155240A1 true US20130155240A1 (en) 2013-06-20

Family

ID=47295930

Family Applications (2)

Application Number Title Priority Date Filing Date
US13/818,323 Abandoned US20130155240A1 (en) 2011-06-07 2012-05-24 Work vehicle periphery monitoring apparatus
US15/392,458 Active US9956915B2 (en) 2011-06-07 2016-12-28 Dump truck periphery monitoring apparatus

Family Applications After (1)

Application Number Title Priority Date Filing Date
US15/392,458 Active US9956915B2 (en) 2011-06-07 2016-12-28 Dump truck periphery monitoring apparatus

Country Status (6)

Country Link
US (2) US20130155240A1 (ja)
JP (1) JP5597596B2 (ja)
CN (1) CN103079891B (ja)
AU (1) AU2012268486A1 (ja)
CA (1) CA2808260C (ja)
WO (1) WO2012169362A1 (ja)

Cited By (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20140288771A1 (en) * 2011-12-26 2014-09-25 Sumitomo Heavy Industries, Ltd. Image display apparatus for shovel
US20150183370A1 (en) * 2012-09-20 2015-07-02 Komatsu Ltd. Work vehicle periphery monitoring system and work vehicle
US20160111005A1 (en) * 2014-10-15 2016-04-21 Hyundai Motor Company Lane departure warning system and method for controlling the same
US20180319396A1 (en) * 2017-05-02 2018-11-08 Cnh Industrial America Llc Obstacle detection system for a work vehicle
US20180370432A1 (en) * 2017-02-09 2018-12-27 Komatsu Ltd. Work vehicle and display device
US10240323B2 (en) 2014-04-25 2019-03-26 Komatsu Ltd. Surroundings monitoring system, work vehicle, and surroundings monitoring method
US10415211B2 (en) * 2015-02-23 2019-09-17 Komatsu Ltd. Hydraulic excavator having handrail and camera
EP3620332A1 (en) * 2018-09-05 2020-03-11 Volvo Car Corporation Driver assistance system and method for vehicle flank safety
EP4050164A4 (en) * 2019-12-25 2023-01-25 Kobelco Construction Machinery Co., Ltd. WORK ASSISTANCE SERVER AND METHODS FOR SELECTING IMAGING EQUIPMENT
US20230050071A1 (en) * 2020-02-21 2023-02-16 Komatsu Ltd. Work machine and remote control system for work machine

Families Citing this family (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP6339337B2 (ja) * 2013-09-24 2018-06-06 日立建機株式会社 車両周囲移動物体検知システム
KR101566784B1 (ko) * 2014-04-25 2015-11-13 주식회사 영신 산업용 및 건설용 중장비 접근감시 및 작동제어 시스템
KR101998300B1 (ko) * 2014-11-05 2019-07-10 현대건설기계 주식회사 작업환경 모니터링 장치 및 작업환경 모니터링 장치에서 수행되는 작업환경 모니터링 방법
JP6589468B2 (ja) * 2015-09-01 2019-10-16 株式会社タダノ 移動式クレーンの周囲表示装置
CN106772383B (zh) * 2016-12-31 2019-03-22 智车优行科技（北京）有限公司 车辆周围物体的检测呈现方法、装置和系统、车辆
TWI668146B (zh) * 2018-03-30 2019-08-11 帷享科技股份有限公司 Vehicle side safety distance display system
CN110667474B (zh) * 2018-07-02 2021-02-26 北京四维图新科技股份有限公司 通用障碍物检测方法、装置与自动驾驶系统
JP7166108B2 (ja) * 2018-08-31 2022-11-07 株式会社小松製作所 画像処理システム、表示装置、画像処理方法、学習済みモデルの生成方法、および学習用データセット
KR20200042549A (ko) * 2018-10-15 2020-04-24 현대자동차주식회사 차량 및 그 제어방법
JP7365122B2 (ja) * 2019-02-01 2023-10-19 株式会社小松製作所 画像処理システムおよび画像処理方法
WO2021115609A1 (en) 2019-12-12 2021-06-17 Ejzenberg Geoffrey A situational awareness system of a cyber-physical hybrid electric autonomous or semi-autonomous off-highway dump truck for surface mining industry

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20100092042A1 (en) * 2008-10-09 2010-04-15 Sanyo Electric Co., Ltd. Maneuvering assisting apparatus
US20110234802A1 (en) * 2010-03-25 2011-09-29 Fujitsu Ten Limited On-vehicle lighting apparatus
US20110292079A1 (en) * 2010-05-26 2011-12-01 Aisin Seiki Kabushiki Kaisha Parking assist apparatus

Family Cites Families (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP3291884B2 (ja) * 1994-01-26 2002-06-17 いすゞ自動車株式会社 車両後方監視装置
JPH11255018A (ja) * 1998-03-11 1999-09-21 Komatsu Ltd 車両用後方監視カメラの洗浄装置
JP4447186B2 (ja) * 2001-06-15 2010-04-07 株式会社小松製作所 建設機械
JP2005020502A (ja) 2003-06-27 2005-01-20 Nippon Seiki Co Ltd 車両用監視装置
JP2008179940A (ja) * 2005-03-31 2008-08-07 Hitachi Constr Mach Co Ltd 作業機械の周囲監視装置
JP2006341641A (ja) * 2005-06-07 2006-12-21 Nissan Motor Co Ltd 映像表示装置及び映像表示方法
JP2007085091A (ja) * 2005-09-22 2007-04-05 Hitachi Constr Mach Co Ltd 作業現場の安全管理装置及びこれに適用される作業機械
US8157837B2 (en) 2006-03-13 2012-04-17 Pneumrx, Inc. Minimally invasive lung volume reduction device and method
JP4996928B2 (ja) 2007-01-05 2012-08-08 日立建機株式会社 作業機械の周囲監視装置
JP5064976B2 (ja) * 2007-11-12 2012-10-31 クラリオン株式会社 建設・土木作業車両の作業安全監視システム
JP4973564B2 (ja) 2008-03-27 2012-07-11 三菱自動車工業株式会社 車両周辺表示装置
JP2011011620A (ja) * 2009-07-01 2011-01-20 Caterpillar Sarl 作業機械の車両周囲確認装置
JP5410327B2 (ja) * 2010-02-18 2014-02-05 株式会社小松製作所 建設機械の後方視界表示システムおよび方法
JP5386762B2 (ja) * 2010-03-04 2014-01-15 株式会社小松製作所 運搬車両の後方視界表示システム

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20100092042A1 (en) * 2008-10-09 2010-04-15 Sanyo Electric Co., Ltd. Maneuvering assisting apparatus
US20110234802A1 (en) * 2010-03-25 2011-09-29 Fujitsu Ten Limited On-vehicle lighting apparatus
US20110292079A1 (en) * 2010-05-26 2011-12-01 Aisin Seiki Kabushiki Kaisha Parking assist apparatus

Cited By (19)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9909283B2 (en) * 2011-12-26 2018-03-06 Sumitomo Heavy Industries, Ltd. Image display apparatus for shovel
US11072911B2 (en) 2011-12-26 2021-07-27 Sumitomo Heavy Industries, Ltd. Image display apparatus for shovel
US20140288771A1 (en) * 2011-12-26 2014-09-25 Sumitomo Heavy Industries, Ltd. Image display apparatus for shovel
US20150183370A1 (en) * 2012-09-20 2015-07-02 Komatsu Ltd. Work vehicle periphery monitoring system and work vehicle
US9333915B2 (en) * 2012-09-20 2016-05-10 Komatsu Ltd. Work vehicle periphery monitoring system and work vehicle
US10240323B2 (en) 2014-04-25 2019-03-26 Komatsu Ltd. Surroundings monitoring system, work vehicle, and surroundings monitoring method
US20160111005A1 (en) * 2014-10-15 2016-04-21 Hyundai Motor Company Lane departure warning system and method for controlling the same
CN105523041A (zh) * 2014-10-15 2016-04-27 现代自动车株式会社 车道偏离警告系统和用于控制该系统的方法
US9824590B2 (en) * 2014-10-15 2017-11-21 Hyundai Motor Company Lane departure warning system and method for controlling the same
US10988912B2 (en) 2015-02-23 2021-04-27 Komatsu Ltd. Hydraulic excavator
US10415211B2 (en) * 2015-02-23 2019-09-17 Komatsu Ltd. Hydraulic excavator having handrail and camera
US20180370432A1 (en) * 2017-02-09 2018-12-27 Komatsu Ltd. Work vehicle and display device
US10668854B2 (en) * 2017-02-09 2020-06-02 Komatsu Ltd. Work vehicle and display device
US10583832B2 (en) * 2017-05-02 2020-03-10 Cnh Industrial America Llc Obstacle detection system for a work vehicle
US20180319396A1 (en) * 2017-05-02 2018-11-08 Cnh Industrial America Llc Obstacle detection system for a work vehicle
EP3620332A1 (en) * 2018-09-05 2020-03-11 Volvo Car Corporation Driver assistance system and method for vehicle flank safety
US11787334B2 (en) 2018-09-05 2023-10-17 Volvo Car Corporation Driver assistance system and method for vehicle flank safety
EP4050164A4 (en) * 2019-12-25 2023-01-25 Kobelco Construction Machinery Co., Ltd. WORK ASSISTANCE SERVER AND METHODS FOR SELECTING IMAGING EQUIPMENT
US20230050071A1 (en) * 2020-02-21 2023-02-16 Komatsu Ltd. Work machine and remote control system for work machine

Also Published As

Publication number Publication date
CN103079891B (zh) 2015-12-02
JP2012254660A (ja) 2012-12-27
CN103079891A (zh) 2013-05-01
CA2808260A1 (en) 2012-12-13
JP5597596B2 (ja) 2014-10-01
US20170120823A1 (en) 2017-05-04
AU2012268486A1 (en) 2013-02-28
CA2808260C (en) 2015-07-14
WO2012169362A1 (ja) 2012-12-13
US9956915B2 (en) 2018-05-01

Similar Documents

Publication Publication Date Title
US9956915B2 (en) Dump truck periphery monitoring apparatus
JP5779244B2 (ja) 作業機械の周囲監視装置
CN108633293B (zh) 作业车辆以及显示装置
US10048692B2 (en) Vehicle travel system
JP5575703B2 (ja) ダンプトラックの積載量表示装置
JP5529943B2 (ja) 作業車両用周辺監視システム及び作業車両
JP5781978B2 (ja) ダンプトラック
US9291709B2 (en) Dump truck
JP5643272B2 (ja) 作業車両用周辺監視システム及び作業車両
WO2014045457A1 (ja) 作業車両用周辺監視システム及び作業車両
JP5926315B2 (ja) 作業車両用周辺監視システム及び作業車両
WO2014020973A1 (ja) 運搬車両の監視装置
JP6012680B2 (ja) ダンプトラック
JP2014222877A (ja) 作業車両の周辺監視装置
JP5990237B2 (ja) ダンプトラック用周辺監視システム及びダンプトラック
JP5964353B2 (ja) ダンプトラック

Legal Events

Date Code Title Description
AS Assignment

Owner name: KOMATSU LTD, JAPAN

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:MITSUTA, SHINJI;HARADA, SHIGERU;TANUKI, TOMIKAZU;AND OTHERS;SIGNING DATES FROM 20130118 TO 20130128;REEL/FRAME:029855/0450

STCB Information on status: application discontinuation

Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION