WO2014103683A1 - Device to display area around body of working machine - Google Patents

Device to display area around body of working machine Download PDF

Info

Publication number
WO2014103683A1
WO2014103683A1 PCT/JP2013/082985 JP2013082985W WO2014103683A1 WO 2014103683 A1 WO2014103683 A1 WO 2014103683A1 JP 2013082985 W JP2013082985 W JP 2013082985W WO 2014103683 A1 WO2014103683 A1 WO 2014103683A1
Authority
WO
WIPO (PCT)
Prior art keywords
calibration
vehicle body
image
unit
cameras
Prior art date
Application number
PCT/JP2013/082985
Other languages
French (fr)
Japanese (ja)
Inventor
古渡　陽一
慶仁 稲野辺
石本　英史
守飛 太田
Original Assignee
日立建機株式会社
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 日立建機株式会社 filed Critical 日立建機株式会社
Publication of WO2014103683A1 publication Critical patent/WO2014103683A1/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/97—Determining parameters from multiple pictures
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R1/00—Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
    • B60R1/20—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
    • B60R1/22—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles for viewing an area outside the vehicle, e.g. the exterior of the vehicle
    • B60R1/23—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles for viewing an area outside the vehicle, e.g. the exterior of the vehicle with a predetermined field of view
    • B60R1/27—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles for viewing an area outside the vehicle, e.g. the exterior of the vehicle with a predetermined field of view providing all-round vision, e.g. using omnidirectional cameras
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N17/00—Diagnosis, testing or measuring for television systems or their details
    • H04N17/002—Diagnosis, testing or measuring for television systems or their details for television cameras
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
    • B60R2300/10—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of camera system used
    • B60R2300/105—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of camera system used using multiple cameras
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
    • B60R2300/40—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the details of the power supply or the coupling to vehicle components
    • B60R2300/402—Image calibration
    • E—FIXED CONSTRUCTIONS
    • E02—HYDRAULIC ENGINEERING; FOUNDATIONS; SOIL SHIFTING
    • E02F—DREDGING; SOIL-SHIFTING
    • E02F9/00—Component parts of dredgers or soil-shifting machines, not restricted to one of the kinds covered by groups E02F3/00 - E02F7/00
    • E02F9/26—Indicating devices
    • E02F9/261—Surveying the work-site to be treated
    • E—FIXED CONSTRUCTIONS
    • E02—HYDRAULIC ENGINEERING; FOUNDATIONS; SOIL SHIFTING
    • E02F—DREDGING; SOIL-SHIFTING
    • E02F9/00—Component parts of dredgers or soil-shifting machines, not restricted to one of the kinds covered by groups E02F3/00 - E02F7/00
    • E02F9/26—Indicating devices
    • E02F9/267—Diagnosing or detecting failure of vehicles
    • E02F9/268—Diagnosing or detecting failure of vehicles with failure correction follow-up actions
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30204—Marker
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30248—Vehicle exterior or interior
    • G06T2207/30252—Vehicle exterior; Vicinity of vehicle

Definitions

  • the present invention relates to a vehicle body circumference display device for a working machine that captures and displays the surroundings of a working machine such as a dump truck.
  • work machines such as dump trucks and hydraulic shovels are provided with a plurality of cameras for photographing the periphery of the vehicle body, and images taken by these cameras are displayed on the display unit in the cab. ing.
  • the operator in the cab can know the situation around the vehicle body while sitting in the driver's seat by confirming the image displayed on the display unit, so that the work can be performed safely.
  • the monitor apparatus for construction machine displays the normal image of the object to be imaged taken by the rear camera on the right side of the display means, and the image of the object to be imaged taken by the side camera on the left side
  • continuous images are displayed. Therefore, for example, when an object located behind the vehicle body moves from the shooting range of the rear camera to the shooting range of the side camera, the operator in the cab changes the image displayed on the display unit from the rear camera to the side camera. Since the moving object is continuously displayed on the right and left half portions of the display unit without switching, such an object moving relative to the vehicle body on the rear and side of such a vehicle can be displayed on the display unit Can be intuitively grasped from
  • a vehicle body peripheral display device for displaying a video (hereinafter referred to as a bird's eye video for convenience) showing a view of the vehicle body and surrounding conditions from above the vehicle body.
  • the vehicle body peripheral display device when the vehicle body peripheral display device is provided in a work machine, for example, a plurality of cameras are provided in each part of the vehicle body, and the images taken by these cameras have a predetermined height position.
  • the vehicle body icon (hereinafter, referred to as “ ⁇ synthetic surface” for convenience) is displayed on the display unit, and a vehicle body icon indicating the work machine is arranged at the center position of the synthesized image.
  • the operator in the cab can grasp the situation around the vehicle body from the image of the display unit, including the positional relationship between the vehicle body and the objects in the surroundings. Thereby, for example, when the operator in the cab retracts the vehicle body, the vehicle body can be smoothly stopped at the target position while watching the image of the display unit. Moreover, if it is a working machine that can be turned like a hydraulic shovel, the operator can quickly confirm from the image of the display unit that there is nothing that hinders the turning operation around the vehicle body.
  • the above-mentioned vehicle body peripheral display device actually mounts with the mounting position error (predetermined setting position of each camera set up from the mounting position of the camera (for example, indicated by the position coordinates of x, y, z)).
  • Mounting angle error shown by pitch, roll, yaw, etc.
  • a calibration unit is provided which corrects the mounting position error and mounting angle error of the camera and calculates the correction value.
  • the correction of the mounting position error and the mounting angle error of each camera mainly performed by the calibration unit is, for example, a calibration unit that mounts an image taken by each camera after installing a mark as a mark in the shooting range of each camera.
  • the processing is performed by drawing a line (hereinafter referred to as a calibration line for convenience) by designating a mark in the image of the processing unit as a reference point. Then, by reflecting the correction value calculated by the calibration unit, it is possible to display on the display unit an overhead image in which the lap and the shift are suppressed.
  • the mark of the image adjustment method of the prior art is different from the mark serving as the reference point for drawing the calibration line and the application, so the image adjustment method of the prior art is used as it is for correcting the mounting position error and the mounting angle error of the camera. It can not be applied.
  • the present invention has been made from the circumstances of the prior art as described above, and its object is to improve the accuracy of the correction of the mounting position error and the mounting angle error of the camera with respect to the image synthesized by photographing around the vehicle body. It is providing the vehicle body periphery display apparatus of the working machine which can be made to be.
  • a vehicle body periphery display apparatus of a working machine includes a plurality of cameras for photographing the periphery of a vehicle body and a display unit for displaying an image photographed by these cameras.
  • An image composition unit provided in a machine for combining images taken by the respective cameras, and an attachment position error generated from the attachment position of the camera and an attachment angle of the camera with respect to the images combined by the image composition unit
  • a calibration unit that calculates a correction value of an attachment angle error resulting from the processing, and a processing unit that performs a process of reflecting the correction value calculated by the calibration unit on a video synthesized by the video synthesis unit.
  • the calibration unit includes a plurality of calibration sheets used for calculating the correction value and arranged in the imaging range of the plurality of cameras, and the calibration sheets include calibration markers formed by a plurality of lines.
  • the calibration sheet is photographed by the respective cameras by arranging the calibration sheet in the photographing range of the plurality of cameras, so that the correction of the mounting position error and the mounting angle error of each camera
  • a reference point for projecting an image captured by each camera on a processing unit such as a computer connected to the calibration unit, and for drawing a calibration line on the processing unit on a calibration sheet in a calibration sheet in the image can be specified as
  • the image on the processing unit is converted into a bird's-eye view image so that it is easy to specify a reference point for drawing a calibration line, and even if a portion of the calibration marker is not clearly displayed, the calibration marker is displayed.
  • a reference point for drawing a calibration line can be accurately specified on the processing unit. The accuracy of the correction of the mounting position error and the mounting angle error of the camera can be improved.
  • the calibration marker includes an intersection point formed by crossing the plurality of lines. According to this configuration, since the point of intersection of the calibration markers can be selected as a reference point for drawing a calibration line on the processing unit, this reference point can be designated quickly. Thereby, the correction of the mounting position error and the mounting angle error of the camera can be smoothly performed.
  • the calibration marker has a predetermined degree of mutual overlapping of the calibration sheets shown in the image processed by the processing unit. And forming a predetermined pattern as an index for determining whether or not it is within the range of.
  • the image combining unit when the image combining unit combines the images captured by the respective cameras, the calibration markers of the calibration sheet in the images captured by the respective cameras overlap with each other, whereby the processing unit The degree of overlap of the calibration sheets appearing in the processed video can be determined with reference to the predetermined pattern of each calibration marker in the video. As a result, it is possible to quickly grasp the size of the lap and the deviation in the image in which the attachment position error and the attachment angle error of each camera have been corrected, and the confirmation operation of the image after the correction can be easily performed. .
  • the plurality of lines may be formed by a plurality of straight lines
  • the predetermined pattern may be a plurality of squares formed by the plurality of straight lines. It is characterized in that it is formed of a plurality of straight lines, includes two lines intersecting the plurality of squares, and makes an intersection of these two lines common to a center point of the plurality of squares.
  • overlapping calibration is performed by superimposing two intersecting lines of calibration markers in an image captured by each camera and a quadrangle on the same screen in the image combining unit and comparing them with each other. Since it is possible to measure the position of the other relative to one of the motion markers, the degree of mutual overlapping of the images processed by the processing unit is within the predetermined range as originally planned, ie the correction is as intended It can be accurately determined whether it has been completed.
  • the calibration sheet provided in the imaging range of the plurality of cameras is provided, and the calibration marker in the calibration sheet is formed by the plurality of lines.
  • FIG. 8A is a diagram showing an image displayed on the screen of the monitor according to the present embodiment
  • FIG. 8A is a diagram showing an image when correction is not performed on the image synthesized by the image synthesis unit
  • Fig. 11 shows an image when correction is performed on the image synthesized by the image synthesizing unit. It is a figure explaining the amount of wraps and the amount of gap of the picture compounded by the picture synthesis part concerning this embodiment.
  • FIG. 14 is a diagram showing an example when calibration sheets shown in FIG. 13 overlap in the image shown in FIG. 9;
  • a vehicle body periphery display device is provided on a dump truck 1 as a working machine as shown in FIG. 1, for example.
  • the dump truck 1 is provided with, for example, a frame 5, front wheels 6 rotatably provided on both left and right ends of the front of the frame 5, and two wheels rotatably provided on left and right ends of the rear of the frame 5. And a rear wheel 7. Further, the dump truck 1 is provided on the frame 5 so as to be able to be undulated, and is provided with a loading platform 2 for loading a load such as earth and sand or crushed stone.
  • the dump truck 1 is disposed forward of the hinge pin 4 provided at the rear of the frame 5 and the hinge pin 4 of the frame 5, that is, between the front wheel 6 and the rear wheel 7, and the frame 5 and the load carrier
  • the hoist cylinder 3 is connected to the H. 2 to extend the hoist cylinder 3 so as to push up the bed 2 to stand up and to contract while the hoist cylinder 3 is contracted to make it fall while supporting the bed 2 ing.
  • the dump truck 1 transports the load such as earth and sand, crushed stone, etc. loaded on the loading platform 2 in the fallen state, and then shifts the loading platform 2 from the laid-down status to the upright status to tilt the loading platform 2 and unload the load.
  • the dump truck 1 includes a hydraulic pump that supplies pressure oil to the hoist cylinder 3 and a hydraulic oil tank that stores hydraulic oil that supplies the hydraulic pump, and is supplied from the hydraulic pump The hoist cylinder 3 is expanded and contracted by the pressure oil.
  • the dump truck 1 is disposed in front of the loading platform 2 and includes a cab 8 provided on the front wheel 6 side of the frame 5, and the size of the front wheel 6 is larger than the size of the cab 8. Therefore, a step in which the worker acts as a step so that the worker can ascend to the entrance of the cab 8 located above the front wheel 6, for example, the ladder 8 a is hooked on the front side of the cab 8.
  • the dump truck 1 is used to assist the operator in the cab 8
  • four cameras 11a to 11d for photographing the surroundings of the vehicle body and a display unit for displaying an image photographed by these cameras 11a to 11d are provided.
  • the front center of the vehicle body, the front of the left side of the vehicle, the center of the rear of the vehicle, and the front of the right side of the vehicle are arranged (see FIG. 6).
  • the display unit includes, for example, a monitor 15 mounted in the cab 8.
  • the monitor 15 includes, for example, images captured by the cameras 11a to 11d, overhead images obtained by converting these images, and each camera as described later.
  • 11a to 11d are displayed on the screen 16 when the power of the screen 16 is switched on or off and the power switch 18 is on when the power switch 18 is on. It has UP ⁇ DOWN switch 14 which changes the picture.
  • a car body icon 17 representing a car body is disposed at the center of a picture captured and synthesized by each of the cameras 11a to 11d described later among pictures displayed on the screen 16 of the monitor 15.
  • the images taken by the cameras 11a to 11d are taken in, and the operation of the monitor 15 including the switching operation of the image of the screen 16 is controlled according to the signals received from the power switch 18 and the UP / DOWN switch 14.
  • the ECU (electric control unit) 12 is provided.
  • the ECU 12 includes an image combining unit that combines images captured by the cameras 11a to 11d, and an attachment generated from the attachment position of the cameras 11a to 11d with respect to the image combined by the image combining unit.
  • a calibration unit that calculates a correction value of the mounting angle error generated from the position error and the mounting angle of the cameras 11a to 11d.
  • the present embodiment includes a processing unit that performs processing for reflecting the correction value calculated by the calibration unit of the ECU 12 on the video synthesized by the video synthesis unit, and displays the processed video on the screen 16 of the monitor 15 It is like that.
  • the processing unit includes, for example, the above-described ECU 12 and the personal computer 13 connected to the ECU 12, and the above-described processing is performed by the ECU 12 and the personal computer 13.
  • the eyelid combining surface F of the video combining unit is set at a position higher than the ground, for example, at a height position G2 of 1 m from the ground, and the video combining unit sets each camera at this position G2. Images taken at 11a to 11d are synthesized. As a result, as shown in FIG. 5, even if the person 30 enters the boundary portion of the shooting ranges 11A to 11D of the cameras 11a to 11d, the person 30 appears to overlap the image synthesized by the image synthesizing unit. The person 30 can be distinguished from the image of the screen 16 of the monitor 15. As described above, according to the present embodiment, the screen 16 of the monitor 15 is adapted to display the portion in the blind spot from each of the cameras 11a to 11d.
  • the present embodiment includes four calibration sheets 21 used for calculation of correction values by the calibration unit as shown in FIG. 6 and disposed in the imaging ranges 11A to 11D of the four cameras 11a to 11d,
  • These calibration sheets 21 have calibration markers 21A formed of a plurality of lines.
  • the plurality of lines of the calibration marker 21A are, for example, composed of a plurality of straight lines.
  • a black base is used for each calibration sheet 21 and a white tape is attached to each calibration marker 21A, for example.
  • the calibration marker 21A of the calibration sheet 21 includes, for example, two squares 21a and 21b formed by five straight lines as shown in FIG. 7, and these squares 21a and 21b are juxtaposed to form an outer frame It is supposed to be. Further, one of the squares 21a and 21b of the calibration marker 21A is formed by, for example, drawing two diagonal lines 21A1 and crossing three straight lines (two adjacent sides of the square 21a and the diagonal lines 21A1). Includes the four intersection points 21a1. That is, each of the intersections 22a1 coincides with the four apexes 21a1 of the square 21a.
  • the calibration marker 21A forms a predetermined pattern serving as an index for determining whether the degree of mutual overlapping of the calibration sheets 21 shown in the image processed by the processing unit is within the predetermined range.
  • the predetermined pattern includes, for example, two squares 21c and 21d formed by eight straight lines, and two lines 21A1 formed by two straight lines and intersecting the two squares 21c and 21d. The intersection 21a2 of the book line 21A1 and the central point 21a2 of the two squares 21c and 21d are common.
  • the two squares 21c and 21d are, for example, similar to each other, are orthogonal to the two lines 21A1 at a predetermined interval, and the predetermined interval is set to, for example, 500 mm.
  • the two squares 21c and 21d are smaller than the squares 21a and 21b forming the outer frame, and the similarity ratio of the two squares 21c and 21d is, for example, 2: 1.
  • the two lines 21A1 described above coincide with the diagonal lines 21A1 of the squares 21a and 21b of the outer frame, and the square 21a of the outer frame encloses the squares 21c and 21d described above, and the square 21c further squares 21d. It is included.
  • the quadrilaterals 21c and 21d are arranged in a state of being rotated 45 degrees around the center point 21a2 with respect to the square 21a.
  • the operator prepares four calibration sheets 21 arranged around the vehicle body of the dump truck 1, and arranges the four calibration sheets 21 on the ground so as to surround the vehicle body as shown in FIG. For example, the operator arranges two of the four calibration sheets 21 on the side of the camera 11a on the front side of the vehicle body, and arranges the remaining two on the side of the camera 11c on the rear side of the vehicle body Do.
  • the operator arranges the longitudinal direction of each calibration sheet 21 along the longitudinal direction of the vehicle body, and among the squares 21a and 21b of the outer frame of each calibration sheet 21, the square to which the calibration marker 21A is attached.
  • 21a is disposed at a position farther from the vehicle body than the other square 21b.
  • the distance between the calibration sheets 21 on the front side of the vehicle body is the same as the distance between the calibration sheets 21 on the rear side of the vehicle body, and the distance between the calibration sheets 21 on the left side of the vehicle body is The distance between the calibration sheets 21 on the right side of the vehicle body is the same.
  • the calibration sheet 21 on the left side of the two calibration sheets 21 on the front side of the vehicle body is the imaging range 11A of the camera 11a on the front side of the vehicle body and the camera 11b on the left side of the vehicle body
  • the calibration sheet 21 located on the front side of the vehicle body overlaps the imaging range 11A of the camera 11a on the front side of the vehicle body and the right side of the vehicle body. It overlaps with imaging range 11D of camera 11d, and is arranged.
  • the calibration sheet 21 on the left side of the two calibration sheets 21 on the rear side of the vehicle body overlaps the imaging range 11C of the camera 11c on the rear side of the vehicle body and the imaging range 11B on the left side of the vehicle body.
  • the calibration sheet 21 on the right side is the imaging range 11C of the camera 11c on the rear side of the vehicle body and the imaging range of the camera 11d on the right side of the vehicle body It is arranged redundantly to 11D.
  • the image captured by the cameras 11a to 11d is taken into the ECU 12.
  • the ECU 12 temporarily displays the image synthesized by the image synthesizing unit on the screen 16 of the monitor 15 as it is, the mounting position error and the mounting angle error of each of the cameras 11a to 11d as shown in FIG. causes the image on the screen 16 to wrap or shift, so the operator displays the image captured by the ECU 12 on the screen of the personal computer 13 and corrects the mounting position error and the mounting angle error of each of the cameras 11a to 11d. By doing this, the image displayed on the screen 16 of the monitor 15 is corrected.
  • the calibration unit of the ECU 12 calculates correction values of the mounting position error and the mounting angle error of each of the cameras 11a to 11d. Then, the processing unit performs a process of reflecting the correction value on the video synthesized by the video synthesis unit. Next, the operator displays an image on which the correction value is reflected on the screen of the personal computer 13 and checks the image on the screen of the personal computer 13 to determine the degree of overlapping of the calibration sheets 21 shown in the image. Determine if it is within the range of
  • the image projected on the screen of the personal computer 13, that is, the border line of the imaging ranges 11A to 11D of the cameras 11a to 11d on the overhead surface F of the video combining unit in the overhead image Is defined as a mask line, a direction perpendicular to the mask line is defined as a wrap direction, and a direction along the mask line is defined as a shift direction.
  • the degree of overlap of the calibration sheets 21 in the lapping direction of the overhead image is represented as a lapping amount as a relative distance in the lapping direction of the overlapping calibration markers 21A, as shown in FIG.
  • the degree of overlap of the calibration sheets 21 in the offset direction of the overhead image is represented by the relative distance in the offset direction of the overlapping calibration markers 21A as the offset amount. Therefore, when the overlap of the calibration sheets 21 completely matches in the overhead image, the overlap amount and the deviation amount become 0 mm.
  • the lower limit of the wrap amount is set to 500 mm or more so that, for example, those in the boundary portions of the shooting ranges 11A to 11D of the cameras 11a to 11d overlap with the screen 16 of the monitor 15 as the above predetermined range.
  • the upper limit of the lap amount is set to 1500 mm or less in consideration of the deviation amount.
  • the amount of deviation is set to 0 mm or more and 1000 mm or less so that the dead angle of each of the cameras 11a to 11d around the vehicle body can be suppressed.
  • the amount of lap and the amount of deviation are set, and as a result of correcting the attachment position error and the attachment angle error of the cameras 11a to 11d, the lap amount of each calibration sheet 21 in the overhead image displayed on the screen of the personal computer 13 If the amount of deviation falls within the range set as described above, the operator determines that the correction of the mounting position error and the mounting angle error of the cameras 11a to 11d is correctly completed.
  • the operator operates the personal computer 13 to reflect the correction value calculated by the calibration unit of the ECU 12 on the video synthesized by the video synthesis unit, thereby causing laps or misalignment as shown in FIG. 8B.
  • a bird's-eye view image of the surroundings of the vehicle body, of which the has been adjusted, is displayed on the screen 16 of the monitor 15 together with the vehicle body icon 17.
  • the worker mounts the cameras 11a to 11d. It is determined that the correction of the error and the mounting angle error is not completed correctly, and the calibration line is redrawn on the screen of the personal computer 13 or the mounting position and mounting angle of each camera 11a to 11d are adjusted. The correction of the mounting position error and the mounting angle error of the cameras 11a to 11d is executed again, and the process is repeated until it can be determined that the correction is correctly completed.
  • the operator performs calibration on the screen of the personal computer 13 in correcting the mounting position error and the mounting angle error of each of the cameras 11a to 11d with respect to the video synthesized by the video synthesis unit of the ECU 12.
  • the calibration sheet 21 photographed by each of the cameras 11a to 11d is displayed as a bird's eye image on the screen of the personal computer 13. At this time, for example, as shown in FIG.
  • the calibration marker 21A can be expressed on the screen of the personal computer 13 by connecting the portion of the calibration marker 21A of the calibration sheet 21 that can be grasped from the image H.
  • the white calibration marker 21A appears to rise on a black background on the screen of the personal computer 13, the straight line of the calibration marker 21A can be easily seen, and the calibration marker 21A is clearly It can be distinguished. This makes it possible to easily designate a reference point for drawing a calibration line.
  • the calibration marker 21A includes an intersection point 21a1 formed by crossing three straight lines (two adjacent sides of the square 21a and a diagonal line 21A1), that is, four apexes 21a1 of the square 21a.
  • intersections 21a1 can be selected as reference points for drawing a calibration line on the screen of the personal computer 13, this reference point can be designated quickly. Thereby, the correction of the mounting position error and the mounting angle error of the cameras 11a to 11d can be smoothly performed.
  • the image displayed on the screen of the personal computer 13 after the processing by the processing unit is performed as shown in FIG. 10 and FIG. 11 by calibrating the calibration sheet 21 photographed by each of the cameras 11a to 11d. Since the operation markers 21A overlap each other, the appearance of the predetermined pattern of the calibration marker 21A differs according to the overlap amount and the deviation amount of the combined image. Therefore, the operator can quickly grasp the size of the lap and the deviation of the synthesized image from the difference in the appearance of the predetermined pattern. As a result, the operation of checking the image after the correction of the mounting position error and the mounting angle error of each of the cameras 11a to 11d is simplified, so the efficiency of the operation of adjusting the video displayed on the screen 16 of the monitor 15 It can be enhanced.
  • the sides of the respective quadrilaterals 21c and 21d with the two diagonal lines 21A1 intersecting these calibration markers 21A as an axis By using as a scale, it is possible to read the degree of overlap of the calibration sheets 21 from this scale, so it is possible to easily measure the amount of overlap and the amount of shift of the image processed by the processing unit.
  • the degree of mutual overlapping of the calibration sheets 21 shown in the image processed by the processing unit is within the predetermined range as originally planned, that is, the wrap amount is 500 mm or more and 1500 mm or less, and the deviation amount is It is possible to accurately determine whether or not the distance is within the range of 0 mm to 1000 mm, and the accuracy of the correction of the mounting position error and the mounting angle error of the cameras 11a to 11d can be sufficiently improved.
  • a person or the like who has entered a blind spot at the boundary of the imaging ranges 11A to 11D of the cameras 11a to 11d can be reliably displayed on the screen 16 of the monitor 15, thus achieving excellent safety. it can.
  • the rectangular outer frame as a whole is formed by including the two squares 21a and 21b in which the calibration markers 21A of the calibration sheet 21 are arranged in parallel, as shown in FIG. Even if the placement places of the test sheets 21 are apart in the longitudinal direction of the vehicle body, the longitudinal direction of each calibration sheet 21 is the longitudinal direction of the vehicle body simply by aligning the sides of the squares 21a and 21b of the calibration markers 21A with a measure or the like. Can be precisely placed along the
  • the present embodiment has described the case where the calibration sheet 21 includes the calibration marker 21A formed by a plurality of straight lines, the present invention is not limited to this case.
  • the present invention is not limited to this case.
  • FIG. May have a calibration marker 31A formed of a plurality of curves.
  • the degree of overlapping of the calibration sheets 31 in the image processed by the processing unit can be determined with reference to the predetermined pattern of each calibration marker 31A in the image. The same function and effect as those described above can be obtained.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Mechanical Engineering (AREA)
  • Biomedical Technology (AREA)
  • General Health & Medical Sciences (AREA)
  • Signal Processing (AREA)
  • Health & Medical Sciences (AREA)
  • Component Parts Of Construction Machinery (AREA)
  • Closed-Circuit Television Systems (AREA)
  • Testing, Inspecting, Measuring Of Stereoscopic Televisions And Televisions (AREA)

Abstract

Provided is a device to display an area around the body of a working machine capable of improving the accuracy of corrections for a camera mounting position error and mounting angle error for an image obtained by combining images taken around the vehicle body. A device to display an area around the body of a dump truck (1) comprises an image combining unit to combine images captured by cameras (11a to 11d), a calibration unit to calculate the correction value for a mounting position error and a mounting angle error for the cameras (11a to 11d) with respect to the images to be combined by the image combining unit, and a processing unit to apply the correction value calculated by the calibration unit to the images to be combined by the image combining unit, and displays an image processed by the processing unit on a monitor (15). The device is provided with four calibration sheets (21) that are used in the correction value calculations by the calibration unit and that are disposed in an imaging range (11A to 11D) of the cameras (11a to 11d), and each calibration sheet (21) has a calibration marker (21A) formed by a plurality of straight lines.

Description

作業機械の車体周囲表示装置Vehicle body circumference display device for work machine
　本発明は、ダンプトラック等の作業機械の周囲を撮影して表示する作業機械の車体周囲表示装置に関する。 BACKGROUND OF THE INVENTION Field of the Invention The present invention relates to a vehicle body circumference display device for a working machine that captures and displays the surroundings of a working machine such as a dump truck.
　一般に、ダンプトラック及び油圧ショベル等の作業機械には、車体の周囲を撮影する複数のカメラが設けられており、これらのカメラによって撮影された映像はキャブ内の表示部に表示されるようになっている。これにより、キャブ内の作業者は表示部に映し出された映像を確認することにより、運転席に着座したまま車体の周囲の状況を知ることができるので、作業を安全に行うことができる。 In general, work machines such as dump trucks and hydraulic shovels are provided with a plurality of cameras for photographing the periphery of the vehicle body, and images taken by these cameras are displayed on the display unit in the cab. ing. As a result, the operator in the cab can know the situation around the vehicle body while sitting in the driver's seat by confirming the image displayed on the display unit, so that the work can be performed safely.
　特に、ダンプトラック及び油圧ショベル等の作業機械は、構造上の理由等によりキャブ内の作業者にとって車体の周囲の視界が制限され易いので、作業者が表示部の映像から車体の周囲の状況を正確に把握できる技術がより重要となっている。このようなキャブ内の作業者の視界を補助する従来技術の１つとして、車体の後方を撮影する後方カメラと、車体の側方を撮影する側方カメラと、これらの後方カメラと側方カメラで撮影された映像を表示するディスプレー手段とを有する建設機械用モニター装置が知られている（例えば、特許文献１参照）。 In particular, work machines such as dump trucks and hydraulic shovels are likely to restrict the view around the vehicle body for the operator in the cab due to structural reasons etc. Technologies that can be accurately grasped are becoming more important. As one of the conventional techniques for assisting the view of the operator in such a cab, a rear camera for shooting the rear of the vehicle body, a side camera for shooting the side of the vehicle body, and these rear cameras and side cameras There is known a monitor device for construction equipment having a display means for displaying an image photographed by the above (see, for example, Patent Document 1).
　この従来技術の建設機械用モニター装置は、後方カメラによって撮影された撮像対象の正像をディスプレー手段の右側半部に表示し、側方カメラによって撮影された撮像対象の正像を左側半部に表示することにより、連続する画像を表示するようにしている。従って、例えば車体の後方に位置する物体が後方カメラの撮影範囲から側方カメラの撮影範囲に移動したときには、キャブ内の作業者は、表示部に表示された映像を後方カメラから側方カメラに切替えなくても、移動する物体が表示部の右側半部と左側半部で連続して表示されるので、このような車体の後方及び側方で車体に対して相対的に動く物体を表示部の映像から直感的に把握することができる。 The monitor apparatus for construction machine according to the prior art displays the normal image of the object to be imaged taken by the rear camera on the right side of the display means, and the image of the object to be imaged taken by the side camera on the left side By displaying, continuous images are displayed. Therefore, for example, when an object located behind the vehicle body moves from the shooting range of the rear camera to the shooting range of the side camera, the operator in the cab changes the image displayed on the display unit from the rear camera to the side camera. Since the moving object is continuously displayed on the right and left half portions of the display unit without switching, such an object moving relative to the vehicle body on the rear and side of such a vehicle can be displayed on the display unit Can be intuitively grasped from
　また、キャブ内の作業者の視界を補助する他の従来技術の１つとして、近年では車体に搭載された複数のカメラで車体の周囲を撮影し、撮影した映像を応用することで表示部に車体の上空から車体と周囲の状況を俯瞰した映像（以下、便宜的に俯瞰映像と呼ぶ）を表示させる車体周囲表示装置が提案されている。 In addition, as one of other conventional techniques for assisting the operator in the cab, in recent years, the surroundings of the vehicle body are photographed with a plurality of cameras mounted on the vehicle body, and the photographed images are applied to the display unit There has been proposed a vehicle body peripheral display device for displaying a video (hereinafter referred to as a bird's eye video for convenience) showing a view of the vehicle body and surrounding conditions from above the vehicle body.
　具体的には、この車体周囲表示装置が作業機械に設けられた場合には、例えば車体の各部分に複数のカメラが設けられており、これらのカメラで撮影された映像が所定の高さ位置の面（以下、便宜的に俯瞰合成面と呼ぶ）上で合成して表示部に表示され、作業機械を示す車体アイコンが合成された映像の中央位置に配置されるようになっている。 Specifically, when the vehicle body peripheral display device is provided in a work machine, for example, a plurality of cameras are provided in each part of the vehicle body, and the images taken by these cameras have a predetermined height position. The vehicle body icon (hereinafter, referred to as “俯瞰 synthetic surface” for convenience) is displayed on the display unit, and a vehicle body icon indicating the work machine is arranged at the center position of the synthesized image.
　従って、キャブ内の作業者は、表示部の映像から車体と周囲にある物体との位置関係も含めて車体の周囲の状況を把握することができる。これにより、例えばキャブ内の作業者が車体を後退させる際に、表示部の映像を見ながら車体を目的の位置に円滑に停止させることができる。また、油圧ショベルのように旋回可能な作業機械であれば、作業者は表示部の映像から車体の周囲に旋回動作を妨げるものがないことを迅速に確認することができる。 Therefore, the operator in the cab can grasp the situation around the vehicle body from the image of the display unit, including the positional relationship between the vehicle body and the objects in the surroundings. Thereby, for example, when the operator in the cab retracts the vehicle body, the vehicle body can be smoothly stopped at the target position while watching the image of the display unit. Moreover, if it is a working machine that can be turned like a hydraulic shovel, the operator can quickly confirm from the image of the display unit that there is nothing that hinders the turning operation around the vehicle body.
　ここで、複数の映像を合成して表示部に表示する際には、例えば特許文献２に示す従来技術の画像調整方法のように各映像が重なり合った部分における輝度を調整する等の各種の補正が行われるが、上述した車体周囲表示装置は、カメラの取付位置（例えば、ｘ、ｙ、ｚの位置座標で示される）から生じる取付位置誤差（予め設定した各カメラの取付位置と実際に取付けられた取付位置との差）とカメラの取付角度（例えば、ピッチ、ロール、ヨーで示される）から生じる取付角度誤差（予め設定した各カメラの取付角度と実際に取付けられた取付角度との差）により合成した映像にラップやずれが発生するので、カメラの取付位置誤差及び取付角度誤差を補正し、その補正値を算出するキャリブレーション部を備えている。 Here, when a plurality of videos are combined and displayed on the display unit, various corrections, such as adjusting the luminance at the overlapping portion of each video as in the image adjustment method of the prior art shown in Patent Document 2, for example However, the above-mentioned vehicle body peripheral display device actually mounts with the mounting position error (predetermined setting position of each camera set up from the mounting position of the camera (for example, indicated by the position coordinates of x, y, z)). Mounting angle error (shown by pitch, roll, yaw, etc.) and the mounting angle of each camera set in advance and the mounting angle actually mounted. Since a lap and a shift occur in the image synthesized according to the), a calibration unit is provided which corrects the mounting position error and mounting angle error of the camera and calculates the correction value.
　このキャリブレーション部によって主に行われる各カメラの取付位置誤差及び取付角度誤差の補正は、例えば各カメラの撮影範囲に目印となるものを設置した後、各カメラで撮影された映像をキャリブレーション部に接続されたコンピュータ等の処理部に映し出し、処理部の映像中の目印を基準点に指定してライン（以下、便宜的にキャリブレーションラインと呼ぶ）を引くことにより実行される。そして、キャリブレーション部によって算出された補正値を反映させることにより、ラップやずれが抑えられた俯瞰映像を表示部に表示することができる。 The correction of the mounting position error and the mounting angle error of each camera mainly performed by the calibration unit is, for example, a calibration unit that mounts an image taken by each camera after installing a mark as a mark in the shooting range of each camera. The processing is performed by drawing a line (hereinafter referred to as a calibration line for convenience) by designating a mark in the image of the processing unit as a reference point. Then, by reflecting the correction value calculated by the calibration unit, it is possible to display on the display unit an overhead image in which the lap and the shift are suppressed.
特開２００９－７８６０号公報JP, 2009-7860, A 特開平８－２９４０７３号公報Unexamined-Japanese-Patent No. 8-294073
　従来技術の車体周囲表示装置は、各カメラの取付位置誤差及び取付角度誤差を補正する際に通常、コンピュータ等の処理部上でキャリブレーションラインを引くための基準点を指定し易いように、処理部に映し出す映像を俯瞰映像に変換するようにしている。そのため、キャリブレーションラインを引くための基準点となる目印を各カメラで撮影し易いように車体の周囲に配置しても、その目印がカメラから離れた位置に設けられていれば、変換された俯瞰映像の解像度が十分に得られず、俯瞰映像中の画質が部分的に低下する。これにより、処理部上でキャリブレーションラインを引くための基準点を正確に指定できなくなるので、各カメラで撮影して合成された映像に対する補正の精度が低下することが問題となっている。 In the prior art vehicle body peripheral display device, when correcting the mounting position error and the mounting angle error of each camera, processing is usually performed so that it is easy to specify a reference point for drawing a calibration line on a processing unit such as a computer It is designed to convert the video projected on the unit into an overhead video. Therefore, even if a mark serving as a reference point for drawing a calibration line is arranged around the vehicle body so as to be easily taken by each camera, the mark is converted if it is provided at a position away from the camera The resolution of the overhead view video can not be obtained sufficiently, and the image quality in the overhead view video is partially degraded. As a result, the reference point for drawing the calibration line can not be specified accurately on the processing unit, and this causes a problem that the accuracy of correction with respect to the image photographed and synthesized by each camera is lowered.
　特に、上述した特許文献２に開示された従来技術の画像調整方法は、例えば合成される映像のうち一方の画像の補正開始点、及び他方の画像の補正終了点にマークをそれぞれ表示するようにしているが、このマークは、映像が重なり合った部分における輝度を調整するときに映像が重なり合った部分と映像が重なり合っていない部分を目視で確認する手段として便宜的に用いられるものであり、カメラで撮影することによって映像に映し出されるものではないので、上述した各カメラの取付位置誤差及び取付角度誤差を補正する際に見られる俯瞰映像中の画質の低下が考慮されていない。従って、従来技術の画像調整方法のマークは、キャリブレーションラインを引くための基準点となる目印と用途が異なるので、従来技術の画像調整方法をカメラの取付位置誤差及び取付角度誤差の補正にそのまま適用することができない。 In particular, in the image adjustment method of the prior art disclosed in Patent Document 2 described above, for example, marks are respectively displayed at the correction start point of one image and the correction end point of the other image of the images to be synthesized. However, this mark is conveniently used as a means to visually confirm the portion where the images overlap and the portion where the images do not overlap when adjusting the brightness in the portion where the images overlap each other. Since the image is not displayed on the image by photographing, the degradation of the image quality in the overhead image seen when correcting the mounting position error and the mounting angle error of each camera described above is not taken into consideration. Therefore, the mark of the image adjustment method of the prior art is different from the mark serving as the reference point for drawing the calibration line and the application, so the image adjustment method of the prior art is used as it is for correcting the mounting position error and the mounting angle error of the camera. It can not be applied.
　本発明は、このような従来技術の実情からなされたもので、その目的は、車体の周囲を撮影して合成された映像に対し、カメラの取付位置誤差及び取付角度誤差の補正の精度を向上させることができる作業機械の車体周囲表示装置を提供することにある。 The present invention has been made from the circumstances of the prior art as described above, and its object is to improve the accuracy of the correction of the mounting position error and the mounting angle error of the camera with respect to the image synthesized by photographing around the vehicle body. It is providing the vehicle body periphery display apparatus of the working machine which can be made to be.
　上記の目的を達成するために、本発明の作業機械の車体周囲表示装置は、車体の周囲を撮影する複数のカメラと、これらのカメラによって撮影された映像を表示する表示部とを備えた作業機械に設けられ、前記各カメラによって撮影された映像を合成する映像合成部と、この映像合成部によって合成される映像に対して、前記カメラの取付位置から生じる取付位置誤差と前記カメラの取付角度から生じる取付角度誤差の補正値を算出するキャリブレーション部と、このキャリブレーション部によって算出された前記補正値を前記映像合成部によって合成される映像に反映させる処理を行う処理部とを備え、この処理部による処理が行われた映像を前記表示部に表示する作業機械の車体周囲表示装置において、前記キャリブレーション部による前記補正値の算出に用いられ、前記複数のカメラの撮影範囲に配置された複数のキャリブレーションシートを備え、これらのキャリブレーションシートは、複数の線で形成されたキャリブレーションマーカを有することを特徴としている。 In order to achieve the above object, a vehicle body periphery display apparatus of a working machine according to the present invention includes a plurality of cameras for photographing the periphery of a vehicle body and a display unit for displaying an image photographed by these cameras. An image composition unit provided in a machine for combining images taken by the respective cameras, and an attachment position error generated from the attachment position of the camera and an attachment angle of the camera with respect to the images combined by the image composition unit A calibration unit that calculates a correction value of an attachment angle error resulting from the processing, and a processing unit that performs a process of reflecting the correction value calculated by the calibration unit on a video synthesized by the video synthesis unit. In the vehicle body peripheral display device of a working machine that displays an image processed by a processing unit on the display unit, the calibration unit The calibration sheet includes a plurality of calibration sheets used for calculating the correction value and arranged in the imaging range of the plurality of cameras, and the calibration sheets include calibration markers formed by a plurality of lines. And
　このように構成した本発明は、キャリブレーションシートが複数のカメラの撮影範囲に配置されることにより、各カメラによってキャリブレーションシートが撮影されるので、各カメラの取付位置誤差及び取付角度誤差の補正において例えば、各カメラで撮影された映像をキャリブレーション部に接続されたコンピュータ等の処理部に映し出し、映像中のキャリブレーションシートにおけるキャリブレーションマーカを処理部上でキャリブレーションラインを引くための基準点として指定することができる。 In the present invention thus configured, the calibration sheet is photographed by the respective cameras by arranging the calibration sheet in the photographing range of the plurality of cameras, so that the correction of the mounting position error and the mounting angle error of each camera In, for example, a reference point for projecting an image captured by each camera on a processing unit such as a computer connected to the calibration unit, and for drawing a calibration line on the processing unit on a calibration sheet in a calibration sheet in the image. Can be specified as
　このとき、キャリブレーションラインを引くための基準点を指定し易いように処理部上の映像が俯瞰映像に変換され、仮にキャリブレーションマーカの一部が明確に表示されなくても、キャリブレーションマーカが線状に形成されていることにより、明確に表示されている部分の線を繋いだり、あるいは映し出された部分を辿ることで推測できるので、キャリブレーションマーカの形状を容易に把握することができる。従って、各カメラと各キャリブレーションシートとの位置に拘わらず、処理部上でキャリブレーションラインを引くための基準点を正確に指定できるので、車体の周囲を撮影して合成された映像に対し、カメラの取付位置誤差及び取付角度誤差の補正の精度を向上させることができる。 At this time, the image on the processing unit is converted into a bird's-eye view image so that it is easy to specify a reference point for drawing a calibration line, and even if a portion of the calibration marker is not clearly displayed, the calibration marker is displayed. By forming in a linear shape, it is possible to infer by connecting a line of a clearly displayed portion or tracing a projected portion, so that the shape of the calibration marker can be easily grasped. Therefore, regardless of the position of each camera and each calibration sheet, a reference point for drawing a calibration line can be accurately specified on the processing unit. The accuracy of the correction of the mounting position error and the mounting angle error of the camera can be improved.
　また、本発明に係る作業機械の車体周囲表示装置は、前記発明において、前記キャリブレーションマーカは、前記複数の線が交差して形成された交点を含むことを特徴としている。このように構成すると、キャリブレーションマーカの交点を処理部上でキャリブレーションラインを引くための基準点として選択できるので、この基準点の指定を迅速に行うことができる。これにより、カメラの取付位置誤差及び取付角度誤差の補正を円滑に実行することができる。 Further, in the vehicle body peripheral display device for a working machine according to the present invention as set forth in the invention described above, the calibration marker includes an intersection point formed by crossing the plurality of lines. According to this configuration, since the point of intersection of the calibration markers can be selected as a reference point for drawing a calibration line on the processing unit, this reference point can be designated quickly. Thereby, the correction of the mounting position error and the mounting angle error of the camera can be smoothly performed.
　また、本発明に係る作業機械の車体周囲表示装置は、前記発明において、前記キャリブレーションマーカは、前記処理部による処理が行われた映像に映る前記各キャリブレーションシートの相互の重なりの程度が所定の範囲内にあるかどうかを判別する指標となる所定のパターンを形成することを特徴としている。 Further, in the vehicle body peripheral display device for a working machine according to the present invention as set forth in the invention described above, the calibration marker has a predetermined degree of mutual overlapping of the calibration sheets shown in the image processed by the processing unit. And forming a predetermined pattern as an index for determining whether or not it is within the range of.
　このように構成した本発明は、映像合成部が各カメラで撮影された映像を合成すると、各カメラで撮影された映像中のキャリブレーションシートのキャリブレーションマーカが相互に重なることにより、処理部による処理が行われた映像に映る各キャリブレーションシートの重なりの程度を映像中の各キャリブレーションマーカの所定のパターンを参考にして判断することができる。これにより、各カメラの取付位置誤差及び取付角度誤差の補正が行われた映像におけるラップやずれの大きさを迅速に把握することができ、補正後の映像の確認作業を容易に行うことができる。 According to the present invention configured as described above, when the image combining unit combines the images captured by the respective cameras, the calibration markers of the calibration sheet in the images captured by the respective cameras overlap with each other, whereby the processing unit The degree of overlap of the calibration sheets appearing in the processed video can be determined with reference to the predetermined pattern of each calibration marker in the video. As a result, it is possible to quickly grasp the size of the lap and the deviation in the image in which the attachment position error and the attachment angle error of each camera have been corrected, and the confirmation operation of the image after the correction can be easily performed. .
　また、本発明に係る作業機械の車体周囲表示装置は、前記発明において、前記複数の線は複数の直線から成り、前記所定のパターンは、前記複数の直線で形成された複数の四角形と、前記複数の直線で形成され、前記複数の四角形と交差する２本の線とを含み、これらの２本の線の交点と前記複数の四角形の中心点とを共通にすることを特徴としている。 Further, in the vehicle body peripheral display device for a working machine according to the present invention, the plurality of lines may be formed by a plurality of straight lines, and the predetermined pattern may be a plurality of squares formed by the plurality of straight lines. It is characterized in that it is formed of a plurality of straight lines, includes two lines intersecting the plurality of squares, and makes an intersection of these two lines common to a center point of the plurality of squares.
　このように構成した本発明は、各カメラで撮影された映像におけるキャリブレーションマーカの交差する２本の線と四角形を映像合成部で同一画面上に重ね合わせて相互に比較することにより、重なり合うキャリブレーションマーカの一方に対する他方の相対的な位置を測定できるので、処理部による処理が行われた映像の相互の重なりの程度が当初の予定通り所定の範囲内に収まっているか、すなわち補正が意図通りに完了したかどうかを正確に判別することができる。 According to the present invention configured as described above, overlapping calibration is performed by superimposing two intersecting lines of calibration markers in an image captured by each camera and a quadrangle on the same screen in the image combining unit and comparing them with each other. Since it is possible to measure the position of the other relative to one of the motion markers, the degree of mutual overlapping of the images processed by the processing unit is within the predetermined range as originally planned, ie the correction is as intended It can be accurately determined whether it has been completed.
　本発明の作業機械の車体周囲表示装置によれば、複数のカメラの撮影範囲に配置された複数のキャリブレーションシートを備え、これらのキャリブレーションシートにおけるキャリブレーションマーカが複数の線で形成されていることにより、カメラの取付位置誤差及び取付角度誤差を補正する際に、各カメラで撮影して変換された俯瞰映像の画質の低下をキャリブレーションマーカで十分に補うことができる。これにより、処理部上でキャリブレーションラインを引くための基準点を正確に指定できるので、車体の周囲を撮影して合成された映像に対し、カメラの取付位置誤差及び取付角度誤差の補正の精度を向上させることができ、従来よりも作業機械に対する高い信頼性を得ることができる。 According to the vehicle body periphery display device of the work machine of the present invention, the calibration sheet provided in the imaging range of the plurality of cameras is provided, and the calibration marker in the calibration sheet is formed by the plurality of lines. As a result, when correcting the mounting position error and the mounting angle error of the camera, it is possible to sufficiently compensate for the deterioration in the image quality of the overhead image captured and converted by each camera by the calibration marker. Thus, the reference point for drawing the calibration line can be accurately specified on the processing unit, so that the accuracy of the correction of the mounting position error and the mounting angle error of the camera with respect to the image synthesized by photographing around the vehicle body Can be improved, and higher reliability can be obtained for the working machine than ever before.
本発明に係る車体周囲表示装置の一実施形態が備えられる作業機械の一例として挙げたダンプトラックを示す側面図である。BRIEF DESCRIPTION OF THE DRAWINGS It is a side view which shows the dump truck mentioned as an example of the working machine with which one Embodiment of the vehicle body periphery display apparatus which concerns on this invention is equipped. 本発明に係る作業機械の車体周囲表示装置の一実施形態の構成を示す図である。It is a figure which shows the structure of one Embodiment of the vehicle body periphery display apparatus of the working machine which concerns on this invention. 本実施形態に係る映像合成部の俯瞰合成面を地面の高さ位置に設定した状態を説明する図である。It is a figure explaining the state which set the eyelid synthetic | combination surface of the imaging | video synthetic | combination part which concerns on this embodiment to the height position of the ground. 本実施形態に係る映像合成部の俯瞰合成面を地面より高い位置に設定した状態を説明する図である。It is a figure explaining the state which set the eyelid synthetic | combination surface of the imaging | video synthetic | combination part which concerns on this embodiment to a position higher than the ground. 本実施形態に係るモニタの画面に表示された映像のうち各カメラの撮影範囲の境界部分を説明する図である。It is a figure explaining the boundary part of the photography range of each camera among the pictures displayed on the screen of the monitor concerning this embodiment. 本実施形態に係るキャリブレーションシートとダンプトラックの位置関係を説明する図である。It is a figure explaining the positional relationship of the calibration sheet | seat and dump truck which concern on this embodiment. 本実施形態に係るキャリブレーションマーカの構成を示す図である。It is a figure which shows the structure of the calibration marker which concerns on this embodiment. 本実施形態に係るモニタの画面に表示された映像を示す図であり、（ａ）図は映像合成部によって合成される映像に対して補正が行われなかった場合の映像を示す図、（ｂ）図は映像合成部によって合成される映像に対して補正が行われた場合の映像を示す図である。FIG. 8A is a diagram showing an image displayed on the screen of the monitor according to the present embodiment, and FIG. 8A is a diagram showing an image when correction is not performed on the image synthesized by the image synthesis unit; Fig. 11 shows an image when correction is performed on the image synthesized by the image synthesizing unit. 本実施形態に係る映像合成部によって合成される映像のラップ量及びずれ量を説明する図である。It is a figure explaining the amount of wraps and the amount of gap of the picture compounded by the picture synthesis part concerning this embodiment. 図９に示す映像のラップ量の所定の範囲を説明する図である。It is a figure explaining the predetermined | prescribed range of the wrap amount of the imaging | video shown in FIG. 図９に示す映像のずれ量の所定の範囲を説明する図である。It is a figure explaining the predetermined | prescribed range of the shift | offset | difference amount of the imaging | video shown in FIG. 図６に示すダンプトラックの左側部側のカメラによって撮影された映像を俯瞰映像に変換してパソコンに表示した図である。It is a figure which converted into a bird's-eye view image the video image | photographed with the camera by the side of the left side of the dump truck shown in FIG. 6, and displayed it on the personal computer. 図７に示すキャリブレーションシートのキャリブレーションマークの他の一例を示す図である。It is a figure which shows another example of the calibration mark of the calibration sheet | seat shown in FIG. 図９に示す映像において図１３に示すキャリブレーションシートが重なり合ったときの一例を示す図である。FIG. 14 is a diagram showing an example when calibration sheets shown in FIG. 13 overlap in the image shown in FIG. 9;
　以下、本発明に係る作業機械の車体周囲表示装置を実施するための形態を図に基づいて説明する。 Hereinafter, a mode for carrying out a vehicle body peripheral display device for a working machine according to the present invention will be described based on the drawings.
　本発明に係る車体周囲表示装置の一実施形態は、例えば図１に示すように作業機械としてダンプトラック１に設けられる。このダンプトラック１は、例えばフレーム５と、このフレーム５の前部の左右両端に回転可能にそれぞれ一輪ずつ設けられた前輪６と、フレーム５の後部の左右両端に回転可能にそれぞれ二輪ずつ設けられた後輪７とを備えている。また、ダンプトラック１は、フレーム５上に起伏可能に設けられ、土砂や砕石等の積荷を積載する荷台２を備えている。 One embodiment of a vehicle body periphery display device according to the present invention is provided on a dump truck 1 as a working machine as shown in FIG. 1, for example. The dump truck 1 is provided with, for example, a frame 5, front wheels 6 rotatably provided on both left and right ends of the front of the frame 5, and two wheels rotatably provided on left and right ends of the rear of the frame 5. And a rear wheel 7. Further, the dump truck 1 is provided on the frame 5 so as to be able to be undulated, and is provided with a loading platform 2 for loading a load such as earth and sand or crushed stone.
　具体的には、ダンプトラック１は、フレーム５の後部に設けられたヒンジピン４と、フレーム５のうちヒンジピン４よりも前方、すなわち前輪６と後輪７との間に配置され、フレーム５と荷台２とを連結するホイストシリンダ３とを備え、ホイストシリンダ３が伸長することにより、荷台２を押し上げて起立させると共に、ホイストシリンダ３が収縮することにより、荷台２を支持しながら倒伏させるようになっている。 Specifically, the dump truck 1 is disposed forward of the hinge pin 4 provided at the rear of the frame 5 and the hinge pin 4 of the frame 5, that is, between the front wheel 6 and the rear wheel 7, and the frame 5 and the load carrier The hoist cylinder 3 is connected to the H. 2 to extend the hoist cylinder 3 so as to push up the bed 2 to stand up and to contract while the hoist cylinder 3 is contracted to make it fall while supporting the bed 2 ing.
　従って、ダンプトラック１は、倒伏状態において荷台２に積載された土砂や砕石等の積荷を運搬した後、荷台２を倒伏状態から起立状態へ移行させることにより、荷台２を傾斜させて積荷を降ろしている。なお、ダンプトラック１は、図示されないが、ホイストシリンダ３へ圧油を供給する油圧ポンプと、この油圧ポンプに供給する作動油を貯蔵する作動油タンクとを備えており、油圧ポンプから供給される圧油によってホイストシリンダ３が伸縮するようになっている。 Therefore, the dump truck 1 transports the load such as earth and sand, crushed stone, etc. loaded on the loading platform 2 in the fallen state, and then shifts the loading platform 2 from the laid-down status to the upright status to tilt the loading platform 2 and unload the load. ing. Although not shown, the dump truck 1 includes a hydraulic pump that supplies pressure oil to the hoist cylinder 3 and a hydraulic oil tank that stores hydraulic oil that supplies the hydraulic pump, and is supplied from the hydraulic pump The hoist cylinder 3 is expanded and contracted by the pressure oil.
　さらに、ダンプトラック１は、荷台２の前方に配置され、フレーム５のうち前輪６側に設けられたキャブ８を備えており、前輪６の大きさはキャブ８の大きさよりも大きくなっている。従って、作業者が前輪６の上方に位置するキャブ８の入口まで上れるように作業者が踏台とするステップ、例えば梯子８ａがキャブ８の前側にかけられている。 Furthermore, the dump truck 1 is disposed in front of the loading platform 2 and includes a cab 8 provided on the front wheel 6 side of the frame 5, and the size of the front wheel 6 is larger than the size of the cab 8. Therefore, a step in which the worker acts as a step so that the worker can ascend to the entrance of the cab 8 located above the front wheel 6, for example, the ladder 8 a is hooked on the front side of the cab 8.
　このように、ダンプトラック１の車体は大型であり、キャブ８内の作業者にとって車体の周囲の視界が制限され易いので、ダンプトラック１は、キャブ８内の作業者の視界を補助するために、例えば図２に示すように車体の周囲を撮影する４つのカメラ１１ａ～１１ｄと、これらのカメラ１１ａ～１１ｄによって撮影された映像を表示する表示部とを備えており、４つのカメラ１１ａ～１１ｄは、例えば車体の前部の中央位置、車体の左側部のうち前部側、車体の後部の中央位置、及び車体の右側部のうち前部側にそれぞれ配置されている（図６参照）。 As described above, since the body of the dump truck 1 is large and the operator in the cab 8 is likely to have limited visibility around the body, the dump truck 1 is used to assist the operator in the cab 8 For example, as shown in FIG. 2, four cameras 11a to 11d for photographing the surroundings of the vehicle body and a display unit for displaying an image photographed by these cameras 11a to 11d are provided. For example, the front center of the vehicle body, the front of the left side of the vehicle, the center of the rear of the vehicle, and the front of the right side of the vehicle are arranged (see FIG. 6).
　表示部は、例えばキャブ８内に取付けられたモニタ１５から成り、このモニタ１５は、例えば各カメラ１１ａ～１１ｄで撮影された映像、これらの映像を変換した俯瞰映像、及び後述するように各カメラ１１ａ～１１ｄで撮影して合成された映像を映し出す画面１６と、この画面１６の電源をＯＮ状態又はＯＦＦ状態に切替える電源スイッチ１８と、この電源スイッチ１８がＯＮ状態のときに画面１６に映し出される映像を切替えるＵＰ・ＤＯＷＮスイッチ１４とを有している。なお、モニタ１５の画面１６に映し出された映像のうち後述する各カメラ１１ａ～１１ｄで撮影して合成された映像の中央には、車体を示す車体アイコン１７が配置されている。 The display unit includes, for example, a monitor 15 mounted in the cab 8. The monitor 15 includes, for example, images captured by the cameras 11a to 11d, overhead images obtained by converting these images, and each camera as described later. 11a to 11d are displayed on the screen 16 when the power of the screen 16 is switched on or off and the power switch 18 is on when the power switch 18 is on. It has UP · DOWN switch 14 which changes the picture. A car body icon 17 representing a car body is disposed at the center of a picture captured and synthesized by each of the cameras 11a to 11d described later among pictures displayed on the screen 16 of the monitor 15.
　本実施形態は、各カメラ１１ａ～１１ｄで撮影された映像を取込み、電源スイッチ１８及びＵＰ・ＤＯＷＮスイッチ１４から受信した信号に応じて、画面１６の映像の切替動作を含むモニタ１５の動作を制御するＥＣＵ（エレクトリックコントロールユニット）１２を備えている。このＥＣＵ１２は、例えば図示されないが、各カメラ１１ａ～１１ｄによって撮影された映像を合成する映像合成部と、この映像合成部によって合成される映像に対して、カメラ１１ａ～１１ｄの取付位置から生じる取付位置誤差とカメラ１１ａ～１１ｄの取付角度から生じる取付角度誤差の補正値を算出するキャリブレーション部とを有している。 In this embodiment, the images taken by the cameras 11a to 11d are taken in, and the operation of the monitor 15 including the switching operation of the image of the screen 16 is controlled according to the signals received from the power switch 18 and the UP / DOWN switch 14. The ECU (electric control unit) 12 is provided. For example, although not shown, the ECU 12 includes an image combining unit that combines images captured by the cameras 11a to 11d, and an attachment generated from the attachment position of the cameras 11a to 11d with respect to the image combined by the image combining unit. And a calibration unit that calculates a correction value of the mounting angle error generated from the position error and the mounting angle of the cameras 11a to 11d.
　そして、本実施形態は、ＥＣＵ１２のキャリブレーション部によって算出された補正値を映像合成部によって合成される映像に反映させる処理を行う処理部を備え、処理した映像をモニタ１５の画面１６に表示するようにしている。本実施形態では、処理部は、例えば上述のＥＣＵ１２と、このＥＣＵ１２に接続されたパソコン１３とを含み、これらのＥＣＵ１２及びパソコン１３によって上述の処理が行われるようになっている。 Then, the present embodiment includes a processing unit that performs processing for reflecting the correction value calculated by the calibration unit of the ECU 12 on the video synthesized by the video synthesis unit, and displays the processed video on the screen 16 of the monitor 15 It is like that. In the present embodiment, the processing unit includes, for example, the above-described ECU 12 and the personal computer 13 connected to the ECU 12, and the above-described processing is performed by the ECU 12 and the personal computer 13.
　ここで、車体の周囲では、図３に示すように各カメラ１１ａ～１１ｄの撮影範囲１１Ａ～１１Ｄの境界部分のうち特に上部が死角となっており、この部分は各カメラ１１ａ～１１ｄで撮影されないので、ＥＣＵ１２の映像合成部の俯瞰合成面Ｆが地面Ｇ１の高さ位置に設定され、映像合成部が各カメラ１１ａ～１１ｄで撮影した映像を俯瞰合成面Ｆで合成した場合には、死角となった部分にあるものがモニタ１５の画面１６に映らなくなる。 Here, around the vehicle body, as shown in FIG. 3, among the boundary portions of the photographing ranges 11A to 11D of the cameras 11a to 11d, particularly the upper part is a blind spot, and this portion is not photographed by the cameras 11a to 11d Therefore, when the eyelid combining surface F of the image combining unit of the ECU 12 is set at the height position of the ground G1 and the image combining portion combines the images captured by the cameras 11a to 11d with the eyelid combining surface F, What is in the lost part will not appear on the screen 16 of the monitor 15.
　そこで、本実施形態では、図４に示すように映像合成部の俯瞰合成面Ｆを地面より高い位置、例えば地面から１ｍの高さ位置Ｇ２に設定し、映像合成部がこの位置Ｇ２で各カメラ１１ａ～１１ｄで撮影した映像を合成するようにしている。これにより、図５に示すように、仮に人物３０が各カメラ１１ａ～１１ｄの撮影範囲１１Ａ～１１Ｄの境界部分に進入したとしても、映像合成部によって合成された映像に人物３０が重なって映るので、人物３０をモニタ１５の画面１６の映像から見分けることができる。このように、本実施形態は、各カメラ１１ａ～１１ｄから死角となった部分にあるものをモニタ１５の画面１６に映るようにしている。 Therefore, in the present embodiment, as shown in FIG. 4, the eyelid combining surface F of the video combining unit is set at a position higher than the ground, for example, at a height position G2 of 1 m from the ground, and the video combining unit sets each camera at this position G2. Images taken at 11a to 11d are synthesized. As a result, as shown in FIG. 5, even if the person 30 enters the boundary portion of the shooting ranges 11A to 11D of the cameras 11a to 11d, the person 30 appears to overlap the image synthesized by the image synthesizing unit. The person 30 can be distinguished from the image of the screen 16 of the monitor 15. As described above, according to the present embodiment, the screen 16 of the monitor 15 is adapted to display the portion in the blind spot from each of the cameras 11a to 11d.
　さらに、本実施形態は、図６に示すようにキャリブレーション部による補正値の算出に用いられ、４つのカメラ１１ａ～１１ｄの撮影範囲１１Ａ～１１Ｄに配置された４つのキャリブレーションシート２１を備え、これらのキャリブレーションシート２１は、複数の線で形成されたキャリブレーションマーカ２１Ａを有している。このキャリブレーションマーカ２１Ａの複数の線は、例えば複数の直線から成っている。なお、各キャリブレーションシート２１は、例えば黒色の下地が用いられ、各キャリブレーションマーカ２１Ａは、例えばこの黒色の下地上に白色のテープが貼り付けられて描かれている。 Furthermore, the present embodiment includes four calibration sheets 21 used for calculation of correction values by the calibration unit as shown in FIG. 6 and disposed in the imaging ranges 11A to 11D of the four cameras 11a to 11d, These calibration sheets 21 have calibration markers 21A formed of a plurality of lines. The plurality of lines of the calibration marker 21A are, for example, composed of a plurality of straight lines. For example, a black base is used for each calibration sheet 21 and a white tape is attached to each calibration marker 21A, for example.
　次に、キャリブレーションシート２１のキャリブレーションマーカ２１Ａの構成を図７に基づいて詳細に説明する。 Next, the configuration of the calibration marker 21A of the calibration sheet 21 will be described in detail based on FIG.
　キャリブレーションシート２１のキャリブレーションマーカ２１Ａは、例えば図７に示すように５つの直線で形成された２つの正方形２１ａ，２１ｂを含み、これらの正方形２１ａ，２１ｂが並列して外枠が形成されるようになっている。また、キャリブレーションマーカ２１Ａの正方形２１ａ，２１ｂのうち一方の正方形２１ａは、例えば２本の対角線２１Ａ１が引かれ、３つの直線（正方形２１ａの隣接する２辺と対角線２１Ａ１）が交差してそれぞれ形成された４つの交点２１ａ１を含んでいる。すなわち、これらの各交点２２ａ１は、正方形２１ａの４つの頂点２１ａ１と一致している。 The calibration marker 21A of the calibration sheet 21 includes, for example, two squares 21a and 21b formed by five straight lines as shown in FIG. 7, and these squares 21a and 21b are juxtaposed to form an outer frame It is supposed to be. Further, one of the squares 21a and 21b of the calibration marker 21A is formed by, for example, drawing two diagonal lines 21A1 and crossing three straight lines (two adjacent sides of the square 21a and the diagonal lines 21A1). Includes the four intersection points 21a1. That is, each of the intersections 22a1 coincides with the four apexes 21a1 of the square 21a.
　また、キャリブレーションマーカ２１Ａは、処理部による処理が行われた映像に映る各キャリブレーションシート２１の相互の重なりの程度が所定の範囲内にあるかどうかを判別する指標となる所定のパターンを形成している。この所定のパターンは、例えば８つの直線で形成された２つの四角形２１ｃ，２１ｄと、２つの直線で形成され、２つの四角形２１ｃ，２１ｄと交差する２本の線２１Ａ１とを含み、これらの２本の線２１Ａ１の交点２１ａ２と２つの四角形２１ｃ，２１ｄの中心点２１ａ２とを共通にしている。 Further, the calibration marker 21A forms a predetermined pattern serving as an index for determining whether the degree of mutual overlapping of the calibration sheets 21 shown in the image processed by the processing unit is within the predetermined range. doing. The predetermined pattern includes, for example, two squares 21c and 21d formed by eight straight lines, and two lines 21A1 formed by two straight lines and intersecting the two squares 21c and 21d. The intersection 21a2 of the book line 21A1 and the central point 21a2 of the two squares 21c and 21d are common.
　また、２つの四角形２１ｃ，２１ｄは、例えば互いに相似の関係にあり、２本の線２１Ａ１に所定の間隔で直交しており、この所定の間隔は、例えば５００ｍｍに設定されている。本実施形態では、２つの四角形２１ｃ，２１ｄは外枠を形成する正方形２１ａ，２１ｂよりも小さい正方形から成り、２つの四角形２１ｃ，２１ｄの相似比は、例えば２：１となっている。 The two squares 21c and 21d are, for example, similar to each other, are orthogonal to the two lines 21A1 at a predetermined interval, and the predetermined interval is set to, for example, 500 mm. In the present embodiment, the two squares 21c and 21d are smaller than the squares 21a and 21b forming the outer frame, and the similarity ratio of the two squares 21c and 21d is, for example, 2: 1.
　すなわち、上述の２本の線２１Ａ１は外枠の正方形２１ａ，２１ｂの対角線２１Ａ１と一致しており、外枠の正方形２１ａが上述の四角形２１ｃ，２１ｄを内包し、さらにこの四角形２１ｃが四角形２１ｄを内包している。そして、これらの四角形２１ｃ，２１ｄは、正方形２１ａに対して中心点２１ａ２周りに４５度回転した状態で配置されている。 That is, the two lines 21A1 described above coincide with the diagonal lines 21A1 of the squares 21a and 21b of the outer frame, and the square 21a of the outer frame encloses the squares 21c and 21d described above, and the square 21c further squares 21d. It is included. The quadrilaterals 21c and 21d are arranged in a state of being rotated 45 degrees around the center point 21a2 with respect to the square 21a.
　次に、各カメラ１１ａ～１１ｄで撮影して合成される映像の調整作業について詳細に説明する。 Next, the adjustment operation of the image photographed and synthesized by each of the cameras 11a to 11d will be described in detail.
　まず、作業者はダンプトラック１の車体の周囲に配置するキャリブレーションシート２１を４つ用意し、図６に示すように車体を取り囲むように４つのキャリブレーションシート２１を地面上に配置する。例えば、作業者は４つのキャリブレーションシート２１のうち２つを、車体の前部側のカメラ１１ａの側方に配置し、残りの２つを、車体の後部側のカメラ１１ｃの側方に配置する。 First, the operator prepares four calibration sheets 21 arranged around the vehicle body of the dump truck 1, and arranges the four calibration sheets 21 on the ground so as to surround the vehicle body as shown in FIG. For example, the operator arranges two of the four calibration sheets 21 on the side of the camera 11a on the front side of the vehicle body, and arranges the remaining two on the side of the camera 11c on the rear side of the vehicle body Do.
　このとき、作業者は、各キャリブレーションシート２１の長手方向を車体の長手方向に沿って配置し、各キャリブレーションシート２１の外枠の正方形２１ａ，２１ｂのうちキャリブレーションマーカ２１Ａが付された正方形２１ａを他方の正方形２１ｂよりも車体から離れた位置に配置する。なお、車体よりも前方側の各キャリブレーションシート２１の間隔は、車体よりも後方側の各キャリブレーションシート２１の間隔と同じであり、車体よりも左側方側の各キャリブレーションシート２１の間隔は、車体よりも右側方側の各キャリブレーションシート２１の間隔と同じである。 At this time, the operator arranges the longitudinal direction of each calibration sheet 21 along the longitudinal direction of the vehicle body, and among the squares 21a and 21b of the outer frame of each calibration sheet 21, the square to which the calibration marker 21A is attached. 21a is disposed at a position farther from the vehicle body than the other square 21b. The distance between the calibration sheets 21 on the front side of the vehicle body is the same as the distance between the calibration sheets 21 on the rear side of the vehicle body, and the distance between the calibration sheets 21 on the left side of the vehicle body is The distance between the calibration sheets 21 on the right side of the vehicle body is the same.
　このようにして、車体よりも前方側の２つのキャリブレーションシート２１のうち左側のキャリブレーションシート２１は、車体の前部側のカメラ１１ａの撮影範囲１１Ａと車体の左側部側のカメラ１１ｂの撮影範囲１１Ｂに重複して配置され、車体よりも前方側の２つのキャリブレーションシート２１のうち右側のキャリブレーションシート２１は、車体の前部側のカメラ１１ａの撮影範囲１１Ａと車体の右側部側のカメラ１１ｄの撮影範囲１１Ｄに重複して配置されている。 In this way, the calibration sheet 21 on the left side of the two calibration sheets 21 on the front side of the vehicle body is the imaging range 11A of the camera 11a on the front side of the vehicle body and the camera 11b on the left side of the vehicle body Of the two calibration sheets 21 located on the front side of the vehicle body, the calibration sheet 21 located on the front side of the vehicle body overlaps the imaging range 11A of the camera 11a on the front side of the vehicle body and the right side of the vehicle body. It overlaps with imaging range 11D of camera 11d, and is arranged.
　また、車体よりも後方側の２つのキャリブレーションシート２１のうち左側のキャリブレーションシート２１は、車体の後部側のカメラ１１ｃの撮影範囲１１Ｃと車体の左側部側のカメラ１１ｂの撮影範囲１１Ｂに重複して配置され、車体よりも後方側の２つのキャリブレーションシート２１のうち右側のキャリブレーションシート２１は、車体の後部側のカメラ１１ｃの撮影範囲１１Ｃと車体の右側部側のカメラ１１ｄの撮影範囲１１Ｄに重複して配置されている。 Further, the calibration sheet 21 on the left side of the two calibration sheets 21 on the rear side of the vehicle body overlaps the imaging range 11C of the camera 11c on the rear side of the vehicle body and the imaging range 11B on the left side of the vehicle body. Of the two calibration sheets 21 on the rear side of the vehicle body, the calibration sheet 21 on the right side is the imaging range 11C of the camera 11c on the rear side of the vehicle body and the imaging range of the camera 11d on the right side of the vehicle body It is arranged redundantly to 11D.
　次に、図２に示すように作業者は各カメラ１１ａ～１１ｄを起動すると、これらのカメラ１１ａ～１１ｄで撮影した映像がＥＣＵ１２内に取込まれる。ここで、仮にＥＣＵ１２が映像合成部で合成された映像をそのままモニタ１５の画面１６に表示した場合には、図８（ａ）に示すように各カメラ１１ａ～１１ｄの取付位置誤差及び取付角度誤差が原因で画面１６上の映像にラップやずれが発生するので、作業者はＥＣＵ１２で取込んだ映像をパソコン１３の画面に映し出して各カメラ１１ａ～１１ｄの取付位置誤差及び取付角度誤差の補正を行うことにより、モニタ１５の画面１６に表示する映像を修正する。 Next, as shown in FIG. 2, when the operator activates each of the cameras 11a to 11d, the image captured by the cameras 11a to 11d is taken into the ECU 12. Here, if the ECU 12 temporarily displays the image synthesized by the image synthesizing unit on the screen 16 of the monitor 15 as it is, the mounting position error and the mounting angle error of each of the cameras 11a to 11d as shown in FIG. Causes the image on the screen 16 to wrap or shift, so the operator displays the image captured by the ECU 12 on the screen of the personal computer 13 and corrects the mounting position error and the mounting angle error of each of the cameras 11a to 11d. By doing this, the image displayed on the screen 16 of the monitor 15 is corrected.
　このとき、パソコン１３の画面には、各カメラ１１ａ～１１ｄで撮影した映像がＥＣＵ１２を介して俯瞰映像に変換してから表示されるので、作業者は、パソコン１３を操作してパソコン１３の画面に映し出されたキャリブレーションマーカ２１Ａを基準点に指定して画面上でキャリブレーションラインを引き、各カメラ１１ａ～１１ｄの取付位置誤差及び取付角度誤差の補正を実行する。 At this time, since the images taken by the respective cameras 11a to 11d are converted into overhead images through the ECU 12 and displayed on the screen of the personal computer 13, the operator operates the personal computer 13 to display the screen of the personal computer 13 A calibration line is drawn on the screen by designating the calibration marker 21A displayed on the screen as a reference point, and correction of the mounting position error and the mounting angle error of each of the cameras 11a to 11d is executed.
　次に、ＥＣＵ１２のキャリブレーション部は各カメラ１１ａ～１１ｄの取付位置誤差及び取付角度誤差の補正値を算出する。そして、処理部は、この補正値を映像合成部によって合成される映像に反映させる処理を行う。次に、作業者は、補正値が反映された映像をパソコン１３の画面に映し出し、このパソコン１３の画面上の映像を確認することにより、映像に映る各キャリブレーションシート２１の重なりの程度が所定の範囲内にあるかどうかを判別する。 Next, the calibration unit of the ECU 12 calculates correction values of the mounting position error and the mounting angle error of each of the cameras 11a to 11d. Then, the processing unit performs a process of reflecting the correction value on the video synthesized by the video synthesis unit. Next, the operator displays an image on which the correction value is reflected on the screen of the personal computer 13 and checks the image on the screen of the personal computer 13 to determine the degree of overlapping of the calibration sheets 21 shown in the image. Determine if it is within the range of
　具体的には、図９に示すようにパソコン１３の画面上に映し出された映像、すなわち俯瞰映像のうち映像合成部の俯瞰合成面Ｆにおける各カメラ１１ａ～１１ｄの撮影範囲１１Ａ～１１Ｄの境界線をマスク線、このマスク線に対して垂直な方向をラップ方向、マスク線に沿う方向をずれ方向と定義する。 Specifically, as shown in FIG. 9, the image projected on the screen of the personal computer 13, that is, the border line of the imaging ranges 11A to 11D of the cameras 11a to 11d on the overhead surface F of the video combining unit in the overhead image. Is defined as a mask line, a direction perpendicular to the mask line is defined as a wrap direction, and a direction along the mask line is defined as a shift direction.
　そして、図１０に示すように俯瞰映像のラップ方向における各キャリブレーションシート２１の重なりの程度をラップ量として、重なり合うキャリブレーションマーカ２１Ａのラップ方向への相対的な距離で表し、図１１に示すように俯瞰映像のずれ方向における各キャリブレーションシート２１の重なりの程度をずれ量として、重なり合うキャリブレーションマーカ２１Ａのずれ方向への相対的な距離で表している。従って、俯瞰映像において各キャリブレーションシート２１の重なりが完全に一致したときには、ラップ量とずれ量は０ｍｍになる。 Then, as shown in FIG. 10, the degree of overlap of the calibration sheets 21 in the lapping direction of the overhead image is represented as a lapping amount as a relative distance in the lapping direction of the overlapping calibration markers 21A, as shown in FIG. The degree of overlap of the calibration sheets 21 in the offset direction of the overhead image is represented by the relative distance in the offset direction of the overlapping calibration markers 21A as the offset amount. Therefore, when the overlap of the calibration sheets 21 completely matches in the overhead image, the overlap amount and the deviation amount become 0 mm.
　また、上述の所定の範囲として、例えば各カメラ１１ａ～１１ｄの撮影範囲１１Ａ～１１Ｄの境界部分にあるものがモニタ１５の画面１６に重なって映るようにラップ量の下限を５００ｍｍ以上に設定すると共に、ずれ量を考慮してラップ量の上限を１５００ｍｍ以下に設定する。さらに、例えばラップ量を考慮し、車体周りの各カメラ１１ａ～１１ｄの死角が抑えられるようにずれ量を０ｍｍ以上１０００ｍｍ以下に設定する。 In addition, the lower limit of the wrap amount is set to 500 mm or more so that, for example, those in the boundary portions of the shooting ranges 11A to 11D of the cameras 11a to 11d overlap with the screen 16 of the monitor 15 as the above predetermined range. The upper limit of the lap amount is set to 1500 mm or less in consideration of the deviation amount. Furthermore, in consideration of, for example, the amount of lap, the amount of deviation is set to 0 mm or more and 1000 mm or less so that the dead angle of each of the cameras 11a to 11d around the vehicle body can be suppressed.
　このようにラップ量及びずれ量が設定され、カメラ１１ａ～１１ｄの取付位置誤差及び取付角度誤差を補正した結果、パソコン１３の画面上に映し出された俯瞰映像における各キャリブレーションシート２１のラップ量とずれ量が上述のように設定した範囲にそれぞれ収まっていれば、作業者はカメラ１１ａ～１１ｄの取付位置誤差及び取付角度誤差の補正が正確に完了したと判定する。 Thus, the amount of lap and the amount of deviation are set, and as a result of correcting the attachment position error and the attachment angle error of the cameras 11a to 11d, the lap amount of each calibration sheet 21 in the overhead image displayed on the screen of the personal computer 13 If the amount of deviation falls within the range set as described above, the operator determines that the correction of the mounting position error and the mounting angle error of the cameras 11a to 11d is correctly completed.
　そして、作業者は、パソコン１３を操作してＥＣＵ１２のキャリブレーション部によって算出された補正値を映像合成部で合成される映像に反映させることにより、図８（ｂ）に示すようにラップやずれが調整された車体の周囲の俯瞰映像が車体アイコン１７と共にモニタ１５の画面１６に表示される。 Then, the operator operates the personal computer 13 to reflect the correction value calculated by the calibration unit of the ECU 12 on the video synthesized by the video synthesis unit, thereby causing laps or misalignment as shown in FIG. 8B. A bird's-eye view image of the surroundings of the vehicle body, of which the has been adjusted, is displayed on the screen 16 of the monitor 15 together with the vehicle body icon 17.
　一方、パソコン１３の画面上に映し出された俯瞰映像における各キャリブレーションシート２１のラップ量とずれ量が上述のように設定した範囲にそれぞれ収まっていなければ、作業者はカメラ１１ａ～１１ｄの取付位置誤差及び取付角度誤差の補正が正確に完了していないと判定し、パソコン１３の画面上でキャリブレーションラインを引き直したり、あるいは各カメラ１１ａ～１１ｄの取付位置や取付角度を調整することにより、カメラ１１ａ～１１ｄの取付位置誤差及び取付角度誤差の補正を再度実行して補正が正確に完了したと判定できるまで繰り返す。 On the other hand, if the lap amount and the deviation amount of each calibration sheet 21 in the overhead view image displayed on the screen of the personal computer 13 do not fall within the ranges set as described above, the worker mounts the cameras 11a to 11d. It is determined that the correction of the error and the mounting angle error is not completed correctly, and the calibration line is redrawn on the screen of the personal computer 13 or the mounting position and mounting angle of each camera 11a to 11d are adjusted. The correction of the mounting position error and the mounting angle error of the cameras 11a to 11d is executed again, and the process is repeated until it can be determined that the correction is correctly completed.
　このように構成した本実施形態によれば、ＥＣＵ１２の映像合成部で合成される映像に対する各カメラ１１ａ～１１ｄの取付位置誤差及び取付角度誤差の補正において、作業者がパソコン１３の画面上でキャリブレーションラインを引くための基準点を指定する際に、各カメラ１１ａ～１１ｄで撮影されたキャリブレーションシート２１がパソコン１３の画面に俯瞰映像として映し出される。このとき、例えば図１２に示すように車体の左側部側のカメラ１１ｂの撮影範囲１１Ｂにある２つのキャリブレーションシート２１のうちカメラ１１ｂから遠方側のキャリブレーションシート２１の映像Ｈがぼやけて明確に映っていなくても、このキャリブレーションシート２１のキャリブレーションマーカ２１Ａのうち映像Ｈから把握できる部分を繋ぐことにより、キャリブレーションマーカ２１Ａをパソコン１３の画面上で表現することができる。 According to this embodiment configured in this manner, the operator performs calibration on the screen of the personal computer 13 in correcting the mounting position error and the mounting angle error of each of the cameras 11a to 11d with respect to the video synthesized by the video synthesis unit of the ECU 12. When designating a reference point for drawing a communication line, the calibration sheet 21 photographed by each of the cameras 11a to 11d is displayed as a bird's eye image on the screen of the personal computer 13. At this time, for example, as shown in FIG. 12, the image H of the calibration sheet 21 on the far side from the camera 11b among the two calibration sheets 21 in the shooting range 11B of the camera 11b on the left side of the vehicle body Even if it does not appear, the calibration marker 21A can be expressed on the screen of the personal computer 13 by connecting the portion of the calibration marker 21A of the calibration sheet 21 that can be grasped from the image H.
　特に、本実施形態では、パソコン１３の画面において黒色の背景に白色のキャリブレーションマーカ２１Ａが浮かび上がるように映るので、キャリブレーションマーカ２１Ａの直線が見え易くなっており、キャリブレーションマーカ２１Ａを明確に区別することができる。これにより、キャリブレーションラインを引くための基準点を容易に指定することができる。 In particular, in the present embodiment, since the white calibration marker 21A appears to rise on a black background on the screen of the personal computer 13, the straight line of the calibration marker 21A can be easily seen, and the calibration marker 21A is clearly It can be distinguished. This makes it possible to easily designate a reference point for drawing a calibration line.
　このように、パソコン１３の画面に映し出された俯瞰映像の画質の低下を十分に補うことができるので、各カメラ１１ａ～１１ｄと各キャリブレーションシート２１との位置に拘わらず、パソコン１３の画面上でキャリブレーションラインを引くための基準点を正確に指定することができる。従って、車体の周囲を撮影して合成された映像に対し、カメラ１１ａ～１１ｄの取付位置誤差及び取付角度誤差の補正の精度を向上させることができるので、ダンプトラック１に対する高い信頼性を得ることができる。 As described above, since it is possible to sufficiently compensate for the deterioration in the image quality of the overhead view video displayed on the screen of the personal computer 13, regardless of the positions of the respective cameras 11a to 11d and the respective calibration sheets 21, Can accurately specify a reference point for drawing a calibration line. Therefore, it is possible to improve the accuracy of the correction of the mounting position error and the mounting angle error of the cameras 11a to 11d with respect to the combined video of the surroundings of the vehicle body, so that high reliability for the dump truck 1 is obtained. Can.
　また、本実施形態は、キャリブレーションマーカ２１Ａは、３つの直線（正方形２１ａの隣接する２辺と対角線２１Ａ１）が交差して形成された交点２１ａ１、すなわち正方形２１ａの４つの頂点２１ａ１をそれぞれ含むことにより、これらの交点２１ａ１をパソコン１３の画面上でキャリブレーションラインを引くための基準点として選択できるので、この基準点の指定を迅速に行うことができる。これにより、カメラ１１ａ～１１ｄの取付位置誤差及び取付角度誤差の補正を円滑に実行することができる。 Further, in this embodiment, the calibration marker 21A includes an intersection point 21a1 formed by crossing three straight lines (two adjacent sides of the square 21a and a diagonal line 21A1), that is, four apexes 21a1 of the square 21a. Thus, since these intersections 21a1 can be selected as reference points for drawing a calibration line on the screen of the personal computer 13, this reference point can be designated quickly. Thereby, the correction of the mounting position error and the mounting angle error of the cameras 11a to 11d can be smoothly performed.
　また、本実施形態は、処理部による処理が行われてパソコン１３の画面に映し出される映像には、図１０、図１１に示すように各カメラ１１ａ～１１ｄで撮影されたキャリブレーションシート２１のキャリブレーションマーカ２１Ａが相互に重なっているので、合成された映像のラップ量とずれ量に応じてキャリブレーションマーカ２１Ａの所定のパターンの見え方が異なる。そのため、作業者は、この所定のパターンの見え方の違いから合成された映像のラップやずれの大きさを迅速に把握することができる。これにより、各カメラ１１ａ～１１ｄの取付位置誤差及び取付角度誤差の補正が行われた後の映像の確認作業が簡単になるので、モニタ１５の画面１６に表示する映像の調整作業の効率性を高めることができる。 Further, in the present embodiment, the image displayed on the screen of the personal computer 13 after the processing by the processing unit is performed as shown in FIG. 10 and FIG. 11 by calibrating the calibration sheet 21 photographed by each of the cameras 11a to 11d. Since the operation markers 21A overlap each other, the appearance of the predetermined pattern of the calibration marker 21A differs according to the overlap amount and the deviation amount of the combined image. Therefore, the operator can quickly grasp the size of the lap and the deviation of the synthesized image from the difference in the appearance of the predetermined pattern. As a result, the operation of checking the image after the correction of the mounting position error and the mounting angle error of each of the cameras 11a to 11d is simplified, so the efficiency of the operation of adjusting the video displayed on the screen 16 of the monitor 15 It can be enhanced.
　また、本実施形態は、パソコン１３の画面上において重なり合うキャリブレーションマーカ２１Ａを相互に比較する際に、これらのキャリブレーションマーカ２１Ａの交差する２本の対角線２１Ａ１を軸に各四角形２１ｃ，２１ｄの辺を目盛りとして用いることにより、この目盛りからキャリブレーションシート２１同士の重なりの程度を読取ることができるので、処理部による処理が行われた映像のラップ量とずれ量を容易に測定することができる。 Further, in the present embodiment, when comparing calibration markers 21A overlapping each other on the screen of the personal computer 13, the sides of the respective quadrilaterals 21c and 21d with the two diagonal lines 21A1 intersecting these calibration markers 21A as an axis. By using as a scale, it is possible to read the degree of overlap of the calibration sheets 21 from this scale, so it is possible to easily measure the amount of overlap and the amount of shift of the image processed by the processing unit.
　従って、処理部による処理が行われた映像に映る各キャリブレーションシート２１の相互の重なりの程度が当初の予定通り所定の範囲内に収まっているか、すなわちラップ量が５００ｍｍ以上１５００ｍｍ以内、かつずれ量が０ｍｍ以上１０００ｍｍ以内にあるかどうかを正確に判別することができ、カメラ１１ａ～１１ｄの取付位置誤差及び取付角度誤差の補正の精度を十分に向上させることができる。これにより、例えば各カメラ１１ａ～１１ｄの撮影範囲１１Ａ～１１Ｄの境界部分の死角に進入した人物等をモニタ１５の画面１６上に確実に映すことができるので、優れた安全性を実現することができる。 Therefore, the degree of mutual overlapping of the calibration sheets 21 shown in the image processed by the processing unit is within the predetermined range as originally planned, that is, the wrap amount is 500 mm or more and 1500 mm or less, and the deviation amount is It is possible to accurately determine whether or not the distance is within the range of 0 mm to 1000 mm, and the accuracy of the correction of the mounting position error and the mounting angle error of the cameras 11a to 11d can be sufficiently improved. As a result, for example, a person or the like who has entered a blind spot at the boundary of the imaging ranges 11A to 11D of the cameras 11a to 11d can be reliably displayed on the screen 16 of the monitor 15, thus achieving excellent safety. it can.
　また、本実施形態は、キャリブレーションシート２１のキャリブレーションマーカ２１Ａが並列する２つの正方形２１ａ，２１ｂを含むことにより、全体として長方形の外枠が形成されているので、図６に示すようにキャリブレーションシート２１の配置場所が車体の前後方向において離れていても、各キャリブレーションマーカ２１Ａの正方形２１ａ，２１ｂの辺同士をメジャー等で合わせるだけで各キャリブレーションシート２１の長手方向を車体の長手方向に沿って的確に配置することができる。 Further, in the present embodiment, since the rectangular outer frame as a whole is formed by including the two squares 21a and 21b in which the calibration markers 21A of the calibration sheet 21 are arranged in parallel, as shown in FIG. Even if the placement places of the test sheets 21 are apart in the longitudinal direction of the vehicle body, the longitudinal direction of each calibration sheet 21 is the longitudinal direction of the vehicle body simply by aligning the sides of the squares 21a and 21b of the calibration markers 21A with a measure or the like. Can be precisely placed along the
　なお、本実施形態は、作業機械がダンプトラック１から成る場合について説明したが、この場合に限らず、作業機械は油圧ショベル等から成っていても良い。 In addition, although this embodiment demonstrated the case where a working machine consisted of dump truck 1, not only in this case, a working machine may consist of a hydraulic shovel etc.
　また、本実施形態は、キャリブレーションシート２１は、複数の直線で形成されたキャリブレーションマーカ２１Ａを有する場合について説明したが、この場合に限らず、例えば図１３に示すように、キャリブレーションマーカ３１は、複数の曲線で形成されたキャリブレーションマーカ３１Ａを有しても良い。この場合も、図１４に示すように処理部による処理が行われた映像に映る各キャリブレーションシート３１の重なりの程度を映像中の各キャリブレーションマーカ３１Ａの所定のパターンを参考にして判断できるので、上述したのと同様の作用効果を得ることができる。 Further, although the present embodiment has described the case where the calibration sheet 21 includes the calibration marker 21A formed by a plurality of straight lines, the present invention is not limited to this case. For example, as shown in FIG. May have a calibration marker 31A formed of a plurality of curves. Also in this case, as shown in FIG. 14, the degree of overlapping of the calibration sheets 31 in the image processed by the processing unit can be determined with reference to the predetermined pattern of each calibration marker 31A in the image. The same function and effect as those described above can be obtained.
　１　ダンプトラック
　２　荷台
　３　ホイストシリンダ
　４　ヒンジピン
　５　フレーム
　６　前輪
　７　後輪
　８　キャブ
　８ａ　梯子
　１１Ａ～１１Ｄ　撮影範囲
　１１ａ～１１ｄ　カメラ
　１２　ＥＣＵ
　１３　パソコン
　１４　ＵＰ・ＤＯＷＮスイッチ
　１５　モニタ
　１６　画面
　１７　車体アイコン
　１８　電源スイッチ
　２１，３１　キャリブレーションシート
　２１Ａ，３１Ａ　キャリブレーションマーカ
　２１Ａ１　対角線（２本の線）
　２１ａ，２１ｂ　正方形
　２１ａ１，２１ａ２　交点
　２１ｃ，２１ｄ　四角形
　３０　人物 Reference Signs List 1 dump truck 2 carrier 3 hoist cylinder 4 hinge pin 5 frame 6 front wheel 7 rear wheel 8 cab 8a ladder 11A to 11D shooting range 11a to 11d camera 12 ECU
13 personal computer 14 UP / DOWN switch 15 monitor 16 screen 17 body icon 18 power switch 21, 31 calibration sheet 21A, 31A calibration marker 21A1 diagonal (two lines)
21a, 21b square 21a1, 21a2 intersection point 21c, 21d square 30 person

Claims (4)

  1. 　車体の周囲を撮影する複数のカメラ（１１ａ～１１ｄ）と、これらのカメラ（１１ａ～１１ｄ）によって撮影された映像を表示する表示部（１５）とを備えた作業機械（１）に設けられ、
    　前記各カメラ（１１ａ～１１ｄ）によって撮影された映像を合成する映像合成部（１２）と、この映像合成部（１２）によって合成される映像に対して、前記カメラ（１１ａ～１１ｄ）の取付位置から生じる取付位置誤差と前記カメラ（１１ａ～１１ｄ）の取付角度から生じる取付角度誤差の補正値を算出するキャリブレーション部（１２）と、このキャリブレーション部（１２）によって算出された前記補正値を前記映像合成部（１２）によって合成される映像に反映させる処理を行う処理部（１２，１３）とを備え、この処理部（１２，１３）による処理が行われた映像を前記表示部（１５）に表示する作業機械（１）の車体周囲表示装置において、
    　前記キャリブレーション部（１２）による前記補正値の算出に用いられ、前記複数のカメラ（１１ａ～１１ｄ）の撮影範囲（１１Ａ～１１Ｄ）に配置された複数のキャリブレーションシート（２１，３１）を備え、
    　これらのキャリブレーションシート（２１，３１）は、複数の線（２１ａ～２１ｄ，２１Ａ１）で形成されたキャリブレーションマーカ（２１Ａ，３１Ａ）を有することを特徴とする作業機械（１）の車体周囲表示装置。     A work machine (1) including a plurality of cameras (11a to 11d) for photographing the surroundings of a vehicle body and a display unit (15) for displaying an image photographed by the cameras (11a to 11d);
    An image composition unit (12) for combining images taken by the respective cameras (11a to 11d), and attachment positions of the cameras (11a to 11d) with respect to the images to be composed by the image composition unit (12) A calibration unit (12) for calculating a correction value of an attachment position error resulting from the attachment angle and an attachment angle error caused from the attachment angle of the camera (11a to 11d); and the correction value calculated by the calibration unit (12) And a processing unit (12, 13) for performing processing to be reflected on the video synthesized by the video synthesis unit (12), and the display unit (15) displays the video processed by the processing unit (12, 13). In the vehicle body peripheral display device of the working machine (1) displayed on
    The calibration unit (12) includes a plurality of calibration sheets (21, 31) which are used to calculate the correction value and which are disposed in the imaging ranges (11A to 11D) of the plurality of cameras (11a to 11d). ,
    These calibration sheets (21, 31) have a calibration marker (21A, 31A) formed by a plurality of lines (21a to 21d, 21A1), and the vehicle body peripheral display of the working machine (1) apparatus.
  2. 　請求項１に記載の作業機械（１）の車体周囲表示装置において、
    　前記キャリブレーションマーカ（２１Ａ，３１Ａ）は、
    　前記複数の線（２１ａ～２１ｄ，２１Ａ１）が交差して形成された交点（２１ａ１，２１ａ２）を含むことを特徴とする作業機械（１）の車体周囲表示装置。     In the vehicle body peripheral display device of the work machine (1) according to claim 1,
    The calibration markers (21A, 31A) are
    A vehicle body peripheral display device for a working machine (1), comprising intersection points (21a1, 21a2) formed by intersecting the plurality of lines (21a to 21d, 21A1).
  3. 　請求項１に記載の作業機械（１）の車体周囲表示装置において、
    　前記キャリブレーションマーカ（２１Ａ，３１Ａ）は、前記処理部（１２，１３）による処理が行われた映像に映る前記各キャリブレーションシート（２１，３１）の相互の重なりの程度が所定の範囲内にあるかどうかを判別する指標となる所定のパターンを形成することを特徴とする作業機械（１）の車体周囲表示装置。     In the vehicle body peripheral display device of the work machine (1) according to claim 1,
    In the calibration markers (21A, 31A), the degree of mutual overlapping of the calibration sheets (21, 31) in the image processed by the processing unit (12, 13) is within a predetermined range. A vehicle body peripheral display device for a working machine (1), characterized by forming a predetermined pattern that serves as an index for determining whether there is any.
  4. 　請求項３に記載の作業機械（１）の車体周囲表示装置において、
    　前記複数の線（２１ａ～２１ｄ，２１Ａ１）は複数の直線から成り、
    　前記所定のパターンは、
    　前記複数の直線で形成された複数の四角形（２１ｃ，２１ｄ）と、
    　前記複数の直線で形成され、前記複数の四角形（２１ｃ，２１ｄ）と交差する２本の線（２１Ａ１）とを含み、
    　これらの２本の線（２１Ａ１）の交点（２１ａ２）と前記複数の四角形（２１ｃ，２１ｄ）の中心点（２１ａ２）とを共通にすることを特徴とする作業機械（１）の車体周囲表示装置。     In the vehicle body periphery display device of the work machine (1) according to claim 3,
    The plurality of lines (21a to 21d, 21A1) consist of a plurality of straight lines,
    The predetermined pattern is
    A plurality of quadrilaterals (21c, 21d) formed by the plurality of straight lines;
    And a plurality of lines (21A1) formed by the plurality of straight lines and intersecting the plurality of squares (21c, 21d),
    Vehicle body peripheral display device for working machine (1) characterized in that the intersection (21a2) of these two lines (21A1) and the central point (21a2) of the plurality of squares (21c, 21d) are made common .
PCT/JP2013/082985 2012-12-27 2013-12-09 Device to display area around body of working machine WO2014103683A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2012285890A JP6106430B2 (en) 2012-12-27 2012-12-27 Work machine body periphery display device
JP2012-285890 2012-12-27

Publications (1)

Publication Number Publication Date
WO2014103683A1 true WO2014103683A1 (en) 2014-07-03

Family

ID=51020775

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2013/082985 WO2014103683A1 (en) 2012-12-27 2013-12-09 Device to display area around body of working machine

Country Status (2)

Country Link
JP (1) JP6106430B2 (en)
WO (1) WO2014103683A1 (en)

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2016137016A1 (en) * 2016-03-28 2016-09-01 株式会社小松製作所 Assessment device
CN108780799A (en) * 2016-03-24 2018-11-09 索尼公司 Image pick-up device and electronic equipment
US10235774B1 (en) 2017-11-14 2019-03-19 Caterpillar Inc. Method and system for calibration of an image capturing device mounted on a machine
EP3483830A1 (en) * 2017-11-13 2019-05-15 Yamaha Hatsudoki Kabushiki Kaisha Bird's-eye view image generating device for vessel, and calibration method of the same
US10564270B2 (en) 2016-04-13 2020-02-18 Caterpillar Inc. Methods and systems for calibrating sensors

Families Citing this family (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP6425991B2 (en) * 2014-12-19 2018-11-21 アルパイン株式会社 Towing vehicle surrounding image generating apparatus and method for generating towing vehicle surrounding image
US10337174B2 (en) 2015-07-31 2019-07-02 Komatsu Ltd. Working machine display system, working machine display device, and working machine display for prompt screen transition even when abnormal processing occurs
JP6068710B1 (en) * 2016-05-30 2017-01-25 株式会社ネクスコ・エンジニアリング北海道 Overhead image adjustment apparatus and overhead image adjustment program
JP6799557B2 (en) * 2018-03-28 2020-12-16 日立建機株式会社 Work machine
JP2020045687A (en) * 2018-09-19 2020-03-26 日立建機株式会社 Work machine
JP2020147962A (en) * 2019-03-12 2020-09-17 日立建機株式会社 Ambient display system and work machines equipped with the same

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2003283964A (en) * 2002-03-26 2003-10-03 Olympus Optical Co Ltd Video display apparatus
JP2010244326A (en) * 2009-04-07 2010-10-28 Alpine Electronics Inc In-vehicle circumference image display device
JP2011085403A (en) * 2009-10-13 2011-04-28 Mitsutoyo Corp Method for calibrating amount of offset and machine for measuring properties and condition of surface
JP2011228857A (en) * 2010-04-16 2011-11-10 Clarion Co Ltd Calibration device for on-vehicle camera
JP2012112108A (en) * 2010-11-22 2012-06-14 Hitachi Constr Mach Co Ltd Periphery monitoring device for work machine

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2003283964A (en) * 2002-03-26 2003-10-03 Olympus Optical Co Ltd Video display apparatus
JP2010244326A (en) * 2009-04-07 2010-10-28 Alpine Electronics Inc In-vehicle circumference image display device
JP2011085403A (en) * 2009-10-13 2011-04-28 Mitsutoyo Corp Method for calibrating amount of offset and machine for measuring properties and condition of surface
JP2011228857A (en) * 2010-04-16 2011-11-10 Clarion Co Ltd Calibration device for on-vehicle camera
JP2012112108A (en) * 2010-11-22 2012-06-14 Hitachi Constr Mach Co Ltd Periphery monitoring device for work machine

Cited By (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108780799A (en) * 2016-03-24 2018-11-09 索尼公司 Image pick-up device and electronic equipment
CN108780799B (en) * 2016-03-24 2022-12-16 索尼公司 Image pickup device and electronic apparatus
WO2016137016A1 (en) * 2016-03-28 2016-09-01 株式会社小松製作所 Assessment device
JPWO2016137016A1 (en) * 2016-03-28 2017-04-27 株式会社小松製作所 Evaluation device
US10750082B2 (en) 2016-03-28 2020-08-18 Komatsu Ltd. Evaluation device
US10564270B2 (en) 2016-04-13 2020-02-18 Caterpillar Inc. Methods and systems for calibrating sensors
EP3483830A1 (en) * 2017-11-13 2019-05-15 Yamaha Hatsudoki Kabushiki Kaisha Bird's-eye view image generating device for vessel, and calibration method of the same
US11468596B2 (en) 2017-11-13 2022-10-11 Yamaha Hatsudoki Kabushiki Kaisha Bird's-eye view image generating device for vessel, and calibration method of the same
US10235774B1 (en) 2017-11-14 2019-03-19 Caterpillar Inc. Method and system for calibration of an image capturing device mounted on a machine

Also Published As

Publication number Publication date
JP6106430B2 (en) 2017-03-29
JP2014125865A (en) 2014-07-07

Similar Documents

Publication Publication Date Title
WO2014103683A1 (en) Device to display area around body of working machine
JP5316805B2 (en) In-vehicle camera device image adjustment device and in-vehicle camera device
US9169108B2 (en) Performance line display unit
US20150009329A1 (en) Device for monitoring surroundings of machinery
JP6321977B2 (en) Perimeter monitoring equipment for heavy machinery
US20080309784A1 (en) Camera System And Mechanical Apparatus
JP4555876B2 (en) Car camera calibration method
US6985171B1 (en) Image conversion device for vehicle rearward-monitoring device
WO2013183536A1 (en) Display device for self-propelled industrial machine
US9734413B2 (en) Image generation apparatus and paver operation assistance system
WO2014123228A1 (en) Surroundings monitoring device for slewing-type work machine
US10621743B2 (en) Processing-target image creating device, processing-target image creating method, and operation assisting system
JP4560716B2 (en) Vehicle periphery monitoring system
CN104584540A (en) Environment monitoring device for operating machinery
WO2012157379A1 (en) Device for monitoring area around working machine
US10661712B2 (en) Bird's-eye view image system, bird's-eye view image display method and program
WO2017038183A1 (en) Surrounding display device for traveling crane
JP5052463B2 (en) Stereo image processing device for work machines
JP2011500407A (en) Method for calibrating a device comprising at least one omnidirectional camera and an optical display unit
CN212302504U (en) Panoramic all-round looking system of articulated road roller and articulated road roller
JP2008205928A (en) Driving support system
CN111942391A (en) Hinged engineering machinery, panoramic looking-around system and calibration method thereof
CN212289808U (en) Articulated engineering machinery and panoramic all-round looking system thereof
JP6713884B2 (en) Overhead image system, overhead image display method and program
JP4706896B2 (en) Wide-angle image correction method and vehicle periphery monitoring system

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 13867335

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 13867335

Country of ref document: EP

Kind code of ref document: A1