EP3666977A1 - Strassenmaschine - Google Patents

Strassenmaschine Download PDF

Info

Publication number
EP3666977A1
EP3666977A1 EP18843681.0A EP18843681A EP3666977A1 EP 3666977 A1 EP3666977 A1 EP 3666977A1 EP 18843681 A EP18843681 A EP 18843681A EP 3666977 A1 EP3666977 A1 EP 3666977A1
Authority
EP
European Patent Office
Prior art keywords
image
camera
screed
tractor
road machine
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
EP18843681.0A
Other languages
English (en)
French (fr)
Other versions
EP3666977A4 (de
EP3666977B1 (de
Inventor
Nobuyuki Baba
Kazuaki Hagiwara
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Sumitomo SHI Construction Machinery Co Ltd
Original Assignee
Sumitomo SHI Construction Machinery Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Sumitomo SHI Construction Machinery Co Ltd filed Critical Sumitomo SHI Construction Machinery Co Ltd
Publication of EP3666977A1 publication Critical patent/EP3666977A1/de
Publication of EP3666977A4 publication Critical patent/EP3666977A4/de
Application granted granted Critical
Publication of EP3666977B1 publication Critical patent/EP3666977B1/de
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • E—FIXED CONSTRUCTIONS
    • E01—CONSTRUCTION OF ROADS, RAILWAYS, OR BRIDGES
    • E01C—CONSTRUCTION OF, OR SURFACES FOR, ROADS, SPORTS GROUNDS, OR THE LIKE; MACHINES OR AUXILIARY TOOLS FOR CONSTRUCTION OR REPAIR
    • E01C19/00—Machines, tools or auxiliary devices for preparing or distributing paving materials, for working the placed materials, or for forming, consolidating, or finishing the paving
    • E01C19/48—Machines, tools or auxiliary devices for preparing or distributing paving materials, for working the placed materials, or for forming, consolidating, or finishing the paving for laying-down the materials and consolidating them, or finishing the surface, e.g. slip forms therefor, forming kerbs or gutters in a continuous operation in situ
    • E—FIXED CONSTRUCTIONS
    • E01—CONSTRUCTION OF ROADS, RAILWAYS, OR BRIDGES
    • E01C—CONSTRUCTION OF, OR SURFACES FOR, ROADS, SPORTS GROUNDS, OR THE LIKE; MACHINES OR AUXILIARY TOOLS FOR CONSTRUCTION OR REPAIR
    • E01C2301/00—Machine characteristics, parts or accessories not otherwise provided for
    • E01C2301/14—Extendable screeds
    • E01C2301/16—Laterally slidable screeds

Definitions

  • the invention relates to road machines with a screw that feeds a paving material in an axial direction.
  • Patent Document 1 An image generator that generates an image showing an aerial view of an asphalt finisher and its surrounding area from above and presents the image to an operator of the asphalt finisher has been known (see Patent Document 1).
  • This image generator makes it possible for the operator to intuitively understand the positional relationship between the asphalt finisher and an object in its surrounding area by generating and displaying an image showing the entirety of an area around the asphalt finisher.
  • Patent Document 1 Japanese Patent No. 6029941
  • a space around the front end of hopper wings is the blind area of the hopper wings when viewed from a front camera, a left camera, and a right camera.
  • the above-described configuration is not suitable for causing the operator to check the state of a predetermined local area such as an area in front of a screed where a paving material is accumulated during construction. Therefore, the operator has to check the presence or absence of an object in a space that the operator desires to see, the amount of a paving material in a predetermined local area, etc., directly with her/his eyes.
  • a road machine which is a road machine including a tractor and a screed placed behind the tractor, includes a work apparatus configured to feed a paving material in front of the screed and a display device configured to display a peripheral image that is an image showing the surrounding area of the road machine, and to display a local image in a highlighted manner, the local image being an image showing an area fed with the paving material by the work apparatus.
  • FIGS. 1A through 1C illustrates an example configuration of an asphalt finisher 100 serving as a road machine according to an embodiment of the invention.
  • FIG. 1A shows a side view
  • FIG. 1B shows a plan view
  • FIG. 1C shows a rear view.
  • the asphalt finisher 100 mainly includes a tractor 1, a hopper 2, and a screed 3.
  • the tractor 1 is an apparatus for causing the asphalt finisher 100 to travel and tows the screed 3. According to this embodiment, the tractor 1 rotates two or four wheels using traveling hydraulic motors to move the asphalt finisher 100.
  • the traveling hydraulic motors are supplied with hydraulic oil from a hydraulic pump driven by a prime mover such as a diesel engine to rotate.
  • An operator seat 1S and an operation panel 65 are placed on top of the tractor 1.
  • Image capturing devices 51 (a right camera 51R, a left camera 51L, a front camera 51F, a right auxiliary camera 51V, and a left auxiliary camera 51U) are attached to the left side, the right side, and the front of the tractor 1.
  • a display device 52 is installed at such a position as to make it easy for an operator seated in the operator seat 1S to look at the display device 52.
  • the direction of the hopper 2 as viewed from the tractor 1 is defined as a forward direction (the +X direction), and the direction of the screed 3 as viewed from the tractor 1 is defined as a rearward direction (the -X direction).
  • the +Y direction corresponds to a leftward direction
  • the -Y direction corresponds to a rightward direction.
  • the hopper 2 which is an example of a work apparatus, is a mechanism for receiving a paving material (for example, an asphalt mixture).
  • the work apparatus is an apparatus to feed a paving material in front of the screed 3.
  • the hopper 2 can be opened and closed widthwise of the vehicle by hydraulic cylinders.
  • the asphalt finisher 100 fully opens the hopper 2 to receive a paving material from the bed of a dump truck.
  • the asphalt finisher 100 closes the hopper 2 to gather the paving material near the inner wall of the hopper 2 to the center of the hopper 2, thereby making it possible for a conveyor CV, which is an example of a work apparatus, to feed the paving material to the screed 3.
  • the screed 3 is a mechanism for spreading and smoothing a paving material.
  • the screed 3 can be vertically elevated and lowered and be extended and retracted widthwise of the vehicle by hydraulic cylinders. When extended widthwise of the vehicle, the screed 3 is wider than the tractor 1.
  • the screed 3 includes a front screed 30, a left rear screed 31L, and a right rear screed 31R.
  • the left rear screed 31L and the right rear screed 31R can extend and retract widthwise of the vehicle (Y-axis directions).
  • the left rear screed 31L and the right rear screed 31R which can extend and retract widthwise of the vehicle, are arranged with an offset from each other in a travel direction (X-axis direction). Therefore, compared with the case where there is no offset, the left rear screed 31L and the right rear screed 31R can have a larger width (length in the vehicle width direction) and extend more in the vehicle width direction, thus making it possible to construct a wider new pavement.
  • FIG. 2 schematically illustrates an example configuration of an image generating system SYS installed in the asphalt finisher 100.
  • the image generating system SYS generates an output image based on input images captured by the image capturing devices 51 installed in the asphalt finisher 100.
  • the image generating system SYS mainly includes a controller 50, the image capturing devices 51, the display device 52, a storage device 54, and an input device 55.
  • the controller 50 is, for example, a computer including a CPU, a volatile memory, and a nonvolatile memory.
  • the controller 50 causes the CPU to execute programs corresponding to an output image generating part 50A and a highlighting part 50B to implement functions corresponding to the output image generating part 50A and the highlighting part 50B.
  • the image capturing devices 51 are devices to obtain input images for generating an output image.
  • the image capturing devices 51 are cameras including an imaging device such as a CCD or a CMOS.
  • the image capturing devices 51 are so attached to the tractor 1 as to be able to capture an image of the blind area of the operator seated in the operator seat 1S.
  • the blind area includes, for example, an internal space (especially near the tractor 1) of the hopper 2, a space outside the front end of the hopper 2, and a space near a road surface near the side of the asphalt finisher 100.
  • the image capturing devices 51 may be attached to positions other than the right side, the left side, and the front (for example, the rear) of the tractor 1.
  • a wide-angle lens, a fish-eye lens, or the like may be attached to the image capturing devices 51.
  • the image capturing devices 51 may be either attached to the hopper 2 or attached to the screed 3.
  • the image capturing devices 51 include the front camera 51F, the left camera 51L, the right camera 51R, the left auxiliary camera 51U, and the right auxiliary camera 51V.
  • the front camera 51F is attached to the upper end of the front of the tractor 1 such that an optical axis 51FX thereof extends forward in the travel direction and forms an angle ⁇ with the road surface as viewed from the side.
  • the left camera 51L is attached to the upper end of the left side of the tractor 1 such that an optical axis 51LX thereof forms an angle ⁇ with the left side surface of the tractor 1 as viewed from above and forms an angle ⁇ with the road surface as viewed from the rear.
  • the right camera 51R is attached in the same manner as the left camera 51L with right and left reversed.
  • the left auxiliary camera 51U is attached to the upper end of the left side of the tractor 1 such that an optical axis 51UX thereof forms an angle ⁇ with the left side surface of the tractor 1 as viewed from above and forms an angle ⁇ with the road surface as viewed from the rear.
  • the right auxiliary camera 51V is attached in the same manner as the left auxiliary camera 51U with right and left reversed.
  • FIG. 1B an area 51FA surrounded by a dashed line in FIG.
  • an area 51LA surrounded by a one-dot chain line indicates the imaging range of the left camera 51L
  • an area 51RA surrounded by a one-dot chain line indicates the imaging range of the right camera 51R.
  • An area 51UA surrounded by a two-dot chain line indicates the imaging range of the left auxiliary camera 51U
  • an area 51VA surrounded by a two-dot chain line indicates the imaging range of the right auxiliary camera 51V.
  • the left camera 51L and the left auxiliary camera 51U are attached to the tractor 1 such that the area 51UA indicating the imaging range of the left auxiliary camera 51U is completely included in the area 51LA indicating the imaging range of the left camera 51L.
  • the left camera 51L and the left auxiliary camera 51U may be attached to the tractor 1 such that the area 51LA and the area 51UA overlap each other, namely, the area 51UA protrudes from the area 51LA.
  • the right camera 51R and the right auxiliary camera 51V are attached to the tractor 1 such that the area 51VA indicating the imaging range of the right auxiliary camera 51V is completely included in the area 51RA indicating the imaging range of the right camera 51R.
  • the right camera 51R and the right auxiliary camera 51V may be attached to the tractor 1 such that the area 51RA and the area 51VA overlap each other, namely, the area 51VA protrudes from the area 51RA.
  • the left auxiliary camera 51U and the right auxiliary camera 51V may be omitted.
  • the image capturing devices 51 are attached to the asphalt finisher 100 via, for example, a bracket, a stay, a bar, or the like. According to this embodiment, the image capturing devices 51 are attached to the tractor 1 via an attachment stay. Alternatively, the image capturing devices 51 may be directly attached to the tractor 1 without using an attachment stay or be embedded in the tractor 1.
  • the image capturing devices 51 output captured input images to the controller 50.
  • the image capturing devices 51 may output corrected input images in which an apparent distortion or tilt/shift caused by using such a lens is corrected to the controller 50.
  • the image capturing devices 51 may output input images in which the apparent distortion or tilt/shift is not corrected as they are to the controller 50. In this case, the apparent distortion or tilt/shift is corrected by the controller 50.
  • the image capturing devices 51 are arranged such that multiple blind areas to the left and to the right of the asphalt finisher 100 and inside and outside the hopper 2 are included in their imaging ranges.
  • the input device 55 is a device for enabling the operator to input various kinds of information to the image generating system SYS, and is, for example, a touchscreen, buttons, switches, or the like. According to this embodiment, the input device 55 includes a display change switch and a screw dial.
  • the display change switch is a switch for changing configurations of an output image displayed on the display device 52.
  • the screw dial is a dial for controlling the rotational speed of a screw SC that is an example of a work apparatus.
  • the storage device 54 is a device for storing various kinds of information. According to this embodiment, the storage device 54 is a nonvolatile storage device and is integrated into the controller 50. Alternatively, the storage device 54 may be placed outside the controller 50 as a structure different from the controller 50.
  • the display device 52 is a device for displaying various kinds of information. According to this embodiment, the display device 52 is a liquid crystal display installed in the operation panel 65, and displays various images output by the controller 50.
  • the output image generating part 50A is a functional element for generating an output image, and is composed of, for example, software, hardware, or their combination.
  • the output image generating part 50A refers to an input image-output image correspondence map 54a stored in the storage device 54 to correlate coordinates in input image planes in which input images captured by the image capturing devices 51 are positioned with coordinates in an output image plane in which an output image is positioned. Then, the output image generating part 50A generates the output image by associating the values (for example, luminance value, hue value, chromatic value, etc.) of pixels in the output image and the values of pixels in the input images.
  • the values for example, luminance value, hue value, chromatic value, etc.
  • the input image-output image correspondence map 54a stores the correspondence between the coordinates in the input image planes and the coordinates in the output image plane in such a manner as to allow reference to the correspondence.
  • the correspondence is preset based on various parameters of the image capturing devices 51, such as an optical center, a focal length, a CCD size, an optical axis direction vector, a camera horizontal direction vector, and a projection method.
  • the correspondence may be set in such a manner as to prevent an apparent distortion or tilt/shift from appearing in the output image if an input image includes the apparent distortion or tilt/shift.
  • a coordinate group forming a non-rectangular area in the input image plane is correlated with a coordinate group forming a rectangular area in the output image plane.
  • the correspondence may be set such that a coordinate group forming a rectangular area in the input image plane corresponds directly to a coordinate group forming a rectangular area in the output image if the apparent distortion or tilt/shift in the input image is already corrected when the input image is obtained.
  • the highlighting part 50B is a functional element for changing the contents of an output image displayed on the display device 52, and is composed of, for example, software, hardware, or their combination. According to this embodiment, the highlighting part 50B switches an output image displayed on the display device 52 between a first output image and a second output image when the display change switch serving as a highlighting switch is depressed. The highlighting part 50B may also switch the first output image to the second output image when the screw dial is operated and thereafter switch the second output image to the first output image when the length of time without screw dial operation (non-operating time) reaches a predetermined period.
  • the highlighting part 50B may also switch the first output image to the second output image when the highlighting switch is operated and thereafter switch the second output image to the first output image when the length of time without highlighting switch operation (non-operating time) reaches a predetermined period.
  • the non-operating time is counted using the timer function of the controller 50, for example.
  • the first output image includes a peripheral image and does not include a local image.
  • the second output image includes a peripheral image and a local image.
  • the peripheral image is an image showing an area surrounding the asphalt finisher 100.
  • the local image is an image showing a predetermined local area associated with the asphalt finisher 100, for example, an image showing an area where a paving material is fed (scattered out) by the screw SC.
  • the area where a paving material is fed by the screw SC is, for example, an area in front of the screed 3 and surrounded by retaining plates 70 (see FIG. 1B ), side plates 71 (see FIG. 1B ), and mold boards 72 (see FIG. 1B ).
  • the operator can check the amount of a paving material enclosed in the local area while remaining seated in the operator seat 1S without moving around on the tractor 1 or twisting her/his body in the operator seat 1S to look into the local area. Furthermore, the operator can check the surroundings and the amount of a paving material enclosed in the local area substantially simultaneously without greatly moving the line of sight.
  • the asphalt finisher 100 in which the image generating system SYS is installed can reduce the operator's fatigue due to the work of checking the enclosed amount. As a result, it is possible to improve safety with respect to the asphalt finisher 100.
  • the local image may be an image showing an area in the hopper 2.
  • the highlighting part 50B displays the local image in a highlighted manner on the display device 52 so that the operator can distinguish between the peripheral image and the local image.
  • the highlighting part 50B displays the local image in a display frame different from a display frame surrounding the peripheral image.
  • the local image may be displayed over the peripheral image in such a manner as to overlap the peripheral image or be displayed at a different position than the peripheral image so as not to overlap the peripheral image.
  • an image part corresponding to the local area in the peripheral image may be enlarged and displayed.
  • at least part of the other image part of the peripheral image may be reduced in size and displayed, or the display of the other image part may be omitted.
  • the display of the local image with another display frame may be omitted.
  • FIG. 3 is an example of the display of the first output image displayed on the display device 52.
  • the first output image mainly includes a hopper image HG, a left peripheral image LG, a right peripheral image RG, and an illustration image 1CG.
  • the hopper image HG, the left peripheral image LG, and the right peripheral image RG compose the peripheral image.
  • the image generating system SYS displays the hopper image HG, the left peripheral image LG, the right peripheral image RG, and the illustration image 1CG at predetermined positions in predetermined size in the first output image so that the operator can understand that the front of the asphalt finisher 100 coincides with the upper side of the screen of the display device 52. This is for causing the operator to intuitively understand the positional relationship between the asphalt finisher 100 and an object in its surrounding area by showing the operator the first output image serving as an overhead view image showing an aerial view of the asphalt finisher 100 and its surrounding area from above.
  • the hopper image HG is generated based on an input image of the front camera 51F.
  • the hopper image HG is an image showing the state of the inside of the hopper 2 as seen when looking down at the hopper 2 from the tractor 1, and is generated by clipping part of the input image of the front camera 51F to be placed in the top center of the first output image.
  • the left peripheral image LG is generated based on an input image of the left camera 51L.
  • the left peripheral image LG is an image showing the state of a left peripheral area to the left of the asphalt finisher 100 in the travel direction as seen when looking down at the left peripheral area from the tractor 1.
  • the left peripheral image LG is generated by clipping part of the input image of the left camera 51L, performing distortion correction thereon, and further performing image rotation thereon, and is placed at the left end of the first output image.
  • the left peripheral image LG includes an image of the left end portion of the screed 3 and an image of the left end portion of the hopper 2.
  • the right peripheral image RG is generated based on an input image of the right camera 51R.
  • the right peripheral image RG is an image showing the state of a right peripheral area to the right of the asphalt finisher 100 in the travel direction as seen when looking down at the right peripheral area from the tractor 1.
  • the right peripheral image RG is generated by clipping part of the input image of the right camera 51R, performing distortion correction thereon, and further performing image rotation thereon, and is placed at the right end of the first output image.
  • the right peripheral image RG includes an image of the right end portion of the screed 3 and an image of the right end portion of the hopper 2.
  • the distortion correction is image processing for correcting an apparent distortion or tilt/shift caused by using a wide-angle lens or the like.
  • the image rotation is image processing for matching the front side of the asphalt finisher 100 in the travel direction (the upper side of the screen of the display device 52) and the respective orientations of the left peripheral image LG and the right peripheral image RG.
  • the correspondence between the coordinates in the input image planes associated with the respective input images of the left camera 51L and the right camera 51R and the coordinates in the output image plane is stored in the input image-output image correspondence map 54a with effects according to the distortion correction and the image rotation being incorporated in advance.
  • the distortion correction and the image rotation may be performed on the hopper image HG.
  • the illustration image 1CG is a computer-generated graphic of the tractor 1, and is so displayed as to enable the operator to understand the position of the tractor 1. According to this embodiment, the illustration image 1CG is placed in the bottom center of the first output image.
  • the display device 52 can display the first output image showing an aerial view of the asphalt finisher 100 and its surrounding area from above.
  • the hopper image HG, the left peripheral image LG, and the right peripheral image RG are adjacently placed as separate independent images.
  • the three images may be synthesized into a single continuous image.
  • image processing may be performed to prevent disappearance of an image of an object in the overlapping area of the imaging range of the front camera 51F and the imaging ranges of the left camera 51L and the right camera 51R.
  • each of the hopper image HG, the left peripheral image LG, and the right peripheral image RG is generated based on an input image captured by a corresponding single camera.
  • Each of the hopper image HG, the left peripheral image LG, and the right peripheral image RG may be generated based on input images captured by two or more cameras.
  • the left peripheral image LG may be generated based on input images captured by the left camera 51L and the left auxiliary camera 51U.
  • the right peripheral image RG may be generated based on input images captured by the right camera 51R and the right auxiliary camera 51V.
  • FIGS. 4A through 4C are examples of the display of the second output image displayed on the display device 52.
  • the second output image mainly includes the hopper image HG, the left peripheral image LG, the right peripheral image RG, the illustration image 1CG, and a local image SG.
  • the local image SG is displayed over the illustration image 1CG.
  • FIG. 4A illustrates a second output image including a right local image SGR showing a right local area that is an area surrounded by the retaining plate 70 (see FIG. 1B ), the side plate 71 (see FIG. 1B ), and the mold board 72 ( FIG. 1B ) on the right side of the tractor 1.
  • FIG. 4B illustrates a second output image including a left local image SGL showing a left local area that is an area surrounded by the retaining plate 70 (see FIG. 1B ), the side plate 71 (see FIG. 1B ), and the mold board 72 ( FIG. 1B ) on the left side of the tractor 1.
  • FIG. 4C illustrates a second output image including the right local image SGR and the left local image SGL.
  • the right local image SGR is generated based on an input image of the right auxiliary camera 51V.
  • the right local image SGR is an image showing the right local area as seen when looking down at the right local area from the tractor 1.
  • the right local image SGR is generated by clipping part of the input image of the right auxiliary camera 51V, performing distortion correction thereon, and further performing image rotation thereon, and is placed along the right end of the illustration image 1CG over the illustration image 1CG.
  • the left local image SGL is generated based on an input image of the left auxiliary camera 51U.
  • the left local image SGL is an image showing the left local area as seen when looking down at the left local area from the tractor 1.
  • the left local image SGL is generated by clipping part of the input image of the left auxiliary camera 51U, performing distortion correction thereon, and further performing image rotation thereon, and is placed along the left end of the illustration image 1CG over the illustration image 1CG.
  • the display change switch may be configured to include a first switch for displaying the second output image including the right local image SGR (see FIG. 4A ) and a second switch for displaying the second output image including the left local image SGL (see FIG. 4B ).
  • the display change switch may be composed only of a switch for displaying the second output image including the right local image SGR and the left local image SGL (see FIG. 4C ).
  • the display change switch may be configured to include the three switches.
  • the screw dial may be configured to include a right dial for controlling the rotational speed of a right screw and a left dial for controlling the rotational speed of a left screw, or may be composed only of a common dial for controlling the rotational speeds of the right and left screws simultaneously, or may be configured to include the three dials.
  • the highlighting part 50B may display the second output image including the right local image SGR (see FIG. 4A ) when the right dial is operated, and display the second output image including the left local image SGR (see FIG. 4B ) when the left dial is operated.
  • the highlighting part 50B may display the second output image including the right local image SGR and the left local image SGL (see FIG. 4C ) when the common dial is operated.
  • a display frame surrounding the local image SG may be displayed differently from display frames surrounding the hopper image HG, the left peripheral image LG, the right peripheral image RG, and the illustration image 1CG.
  • the display frame surrounding the local image SG may be so displayed as to be different in color, line type, thickness, etc., or may be caused to blink.
  • FIG. 5 is a flowchart of the output image generating process.
  • the output image includes the first output image and the second output image.
  • the image generating system SYS repeatedly executes this output image generating process at predetermined control intervals to selectively generate one of the first output image and the second output image.
  • the image generating system SYS may generate both the first output image and the second output image.
  • the output image generating part 50A of the controller 50 correlates the values of coordinates in the output image plane and the values of coordinates in the input image planes (step S1).
  • the output image generating part 50A refers to the input image-output image correspondence map 54a to obtain the values (for example, luminance value, hue value, chromatic value, etc.) of coordinates in the input image planes corresponding to the coordinates in the output image plane, and set the obtained values as the values of the corresponding coordinates in the output image plane.
  • the controller 50 determines whether the values of all the coordinates in the output image plane are correlated with the values of coordinates in the input image planes (step S2).
  • the output image generating part 50A repeats the process of step S1 and step S2.
  • the output image generating part 50A ends the output image generating process of this time.
  • FIG. 6 is a flowchart of the output image switching process.
  • the image generating system SYS repeatedly executes this output image switching process at predetermined control intervals.
  • the highlighting part 50B of the controller 50 determines whether highlighting is turned on (step S11). For example, the highlighting part 50B determines that highlighting is turned on when the display change switch serving as a highlighting switch is depressed while the first output image is displayed on the display device 52. The highlighting part 50B may determine that highlighting is turned on when the screw dial is operated.
  • the highlighting part 50B displays a local image (step S12) in a highlighted manner. For example, the highlighting part 50B switches the first output image (see FIG. 3 ) displayed on the display device 52 to the second output image (see FIGS. 4A through 4C ) to display at least one of the left local image (SGL) and the right local image (SGR) along with a peripheral image.
  • the highlighting part 50B continues to display the first output image on the display device 52 without switching the displayed first output image to the second output image.
  • the highlighting part 50B determines whether highlighting is turned off (step S13). For example, the highlighting part 50B determines that highlighting is turned off when the display change switch serving as a highlighting switch is depressed while the second output image is displayed on the display device 52. The highlighting part 50B may determine that highlighting is turned off when a predetermined time has passed since the completion of the operation of the screw dial. Alternatively, the highlighting part 50B may determine that highlighting is turned off when a predetermined time has passed since the depression of the display change switch.
  • the highlighting part 50B stops displaying the local image in a highlighted manner (step S14). For example, the highlighting part 50B stops displaying the local image in a highlighted manner by switching the second output image displayed on the display device 52 to the first output image.
  • the highlighting part 50B continues to display the second output image on the display device 52 without switching the displayed second output image to the first output image.
  • the image generating system SYS can display a local image on the display device 52 in response to the operator's request.
  • the operator can check the amount of a paving material enclosed in a local area while remaining seated in the operator seat 1S without moving around on the tractor 1 or twisting her/his body in the operator seat 1S to look into the local area.
  • the operator can check the surroundings and the amount of a paving material enclosed in the local area substantially simultaneously without greatly moving the line of sight.
  • the asphalt finisher 100 in which the image generating system SYS is installed can reduce the operator's fatigue due to the work of checking the enclosed amount. As a result, it is possible to improve safety with respect to the asphalt finisher 100.
  • FIG. 7 is another example of the display of the second output image displayed on the display device 52.
  • the second output image of FIG. 7 is different in that the right local image SGR is displayed over not the illustration image 1CG but the right peripheral image RG from, but otherwise equal to, the second output image of FIG. 4A . Therefore, a description of a common portion is omitted, and differences are described in detail.
  • the following description, which is of the right local image SGR, is also applied to the left local image SGL.
  • the right local image SGR is generated based on an input image of the right camera 51R, the same as the right peripheral image RG. Therefore, the right auxiliary camera 51V may be omitted.
  • the right local image SGR may be an image generated based on an input image of the right auxiliary camera 51V.
  • the right local image SGR corresponds to part of the right peripheral image RG.
  • the right peripheral image RG is composed of a first image part RG1 through an eleventh image part RG11
  • the right local image SGR of FIG. 7 corresponds to the ninth image part RG9.
  • the image generating system SYS displays the right local image SGR, which is an image into which the ninth image part RG9 is vertically enlarged, where the seventh image part RG7 through the eleventh image part RG11 have been displayed. That is, the first image part RG1 through the sixth image part RG6 continue to be displayed, while the seventh image part RG7 through the eleventh image part RG11 are concealed by the right local image SGR to be invisible.
  • the image generating system SYS displays the right local image SGR over image parts showing the right local area in the right peripheral image RG. Therefore, the operator can intuitively understand that the right local area is shown in the right local image SGR. Furthermore, by displaying the right local image SGR, the right local area can be enlarged and displayed compared with the case where the right peripheral image RG is displayed. Therefore, it is possible to more clearly show the operator the state of the right local area fed with a paving material by the right screw.
  • FIG. 8 is yet another example of the display of the second output image displayed on the display device 52.
  • the second output image of FIG. 8 is different in that the right local image SGR is displayed to cover not part but the entirety of the right peripheral image RG from, but otherwise equal to, the second output image of FIG. 7 . Therefore, a description of a common portion is omitted, and differences are described in detail.
  • the following description, which is of the right local image SGR, is also applied to the left local image SGL.
  • the right local image SGR corresponds to part of the right peripheral image RG.
  • the right peripheral image RG is composed of the first image part RG1 through the eleventh image part RG11
  • the right local image SGR of FIG. 8 corresponds to the ninth image part RG9.
  • the image generating system SYS displays the right local image SGR, which is an image into which the ninth image part RG9 is vertically enlarged, where the first image part RG1 through the eleventh image part RG11 have been displayed. That is, the first image part RG1 through the eleventh image part RG11 are concealed by the right local image SGR to be invisible.
  • the image generating system SYS displays the right local image SGR over the entirety of the right peripheral image RG including an image part showing the right local area. Therefore, the operator can intuitively understand that the right local area is shown in the right local image SGR. Furthermore, by displaying the right local image SGR sized to the overall vertical length of the display device 52, the right local area can be further enlarged and displayed compared with the case of the second output image of FIG. 7 . Therefore, it is possible to even more clearly show the operator the state of the right local area fed with a paving material by the right screw.
  • FIG. 9 is still another example of the display of the second output image displayed on the display device 52.
  • the second output image of FIG. 9 is different in that the right local image SGR is generated using not part but the entirety of the right peripheral image RG and that an indicator BG is displayed from, but otherwise equal to, the second output image of FIG. 8 . Therefore, a description of a common portion is omitted, and differences are described in detail.
  • the following description, which is of the right local image SGR, is also applied to the left local image SGL.
  • the right local image SGR corresponds to the entirety of the right peripheral image RG.
  • the right local image SGR of FIG. 9 is generated by vertically enlarging or reducing the size of each of the first image part RG1 through the eleventh image part RG11.
  • the right local image SGR of FIG. 9 is composed of images into which the first image part RG1 through the sixth image part RG6 and the eleventh image part RG11 are vertically reduced in size and images into which the seventh image part RG7 through the tenth image part RG10 are vertically enlarged. That is, unlike in the case of FIGS. 7 and 8 , a view shown in the right peripheral image RG continues to be visible even when the right local image SGR is displayed.
  • the indicator BG is a graphic image representing the state of enlargement or reduction of the image parts of the local image SG relative to the corresponding image parts of the peripheral image.
  • a right indicator BGR represents the state of enlargement or reduction of the image parts of the right local image SGR relative to the corresponding image parts of the right peripheral image RG.
  • the right indicator BGR is a vertically elongated bar-shaped indicator composed of eleven rectangular segments corresponding to the first image part RG1 through the eleventh image part RG11, and is displayed at the right end of the screen. When a left indicator is displayed, the left indicator may be displayed at the left end of the screen.
  • the left indicator represents the state of enlargement or reduction of the image parts of the left local image SGL relative to the corresponding image parts of the left peripheral image LG.
  • the bar-shaped indicator indicates that a vertically longer rectangular segment represents a higher enlargement rate and that a vertically shorter rectangular element represents a higher reduction rate.
  • the display of the indicator BG may be omitted.
  • the image generating system SYS displays the right local image SGR over the entirety of the right peripheral image RG including an image part showing the right local area. Therefore, the operator can intuitively understand that the right local area is shown in the right local image SGR. Furthermore, the same as in the case of FIG. 8 , by displaying the right local image SGR over the overall vertical length of the display device 52, the right local area can be enlarged and displayed compared with the case of the second output image of FIG. 7 . Therefore, it is possible to more clearly show the operator the state of the right local area fed with a paving material by the right screw. Furthermore, unlike in the case of FIG.
  • the image generating system SYS can generate an output image that makes it possible for the operator to intuitively understand the positional relationship between the asphalt finisher 100 and a worker working in its surrounding area, etc., based on input images captured by multiple cameras.
  • the image generating system SYS displays the hopper image HG, the left peripheral image LG, the right peripheral image RG, and the illustration image 1CG so that the operator can be presented with an image showing an aerial view of the asphalt finisher 100 and its surrounding area from above. This makes it possible for the operator to visually check a blind area around the asphalt finisher 100 without leaving the operator seat 1S.
  • the image generating system SYS can improve the safety and operability of the asphalt finisher 100.
  • the image generating system SYS can show the operator the amount of a paving material remaining in the hopper 2, the position of a feature (for example, a maintenance hole) in a road surface to be paved, etc.
  • the image generating system SYS can show the operator the position of a worker working around the asphalt finisher 100, etc. Therefore, the operator can look at the display device 52 and check the position of a worker, etc., and thereafter perform various operations such as opening or closing the hopper, extending or retracting the screed, and raising or lowering the screed 3. Furthermore, the operator can suspend various operations or stop the asphalt finisher when finding that the positional relationship between a worker and the hopper, the screed, or a dump truck is dangerous.
  • the image generating system SYS displays a peripheral image that is an image showing the surrounding area of the asphalt finisher 100 and displays a local image that is an image showing an area fed with a paving material by the screw SC in a highlighted manner.
  • Displaying the local image in a highlighted manner includes displaying the local image in a different display frame, enlarging and displaying the local image, and changing the mode of display of the display frame of the local image. According to this process, the image generating system SYS can clearly show the operator the state of a predetermined local area as well as the state of the surrounding area of the asphalt finisher 100.
  • the image generating system SYS may include the right camera 51R serving as a first camera to capture an image of an area to the right of the asphalt finisher 100 and the right auxiliary camera 51V serving as a second camera to capture an image of an area fed with a paving material by the right screw.
  • the right peripheral image RG that is a component of the peripheral image may be generated based on an image captured by the right camera 51R and the right local image SGR composing the local image may be generated based on an image captured by the right auxiliary camera 51V.
  • the image generating system SYS may include the left camera 51L serving as a first camera to capture an image of an area to the left of the asphalt finisher 100 and the left auxiliary camera 51U serving as a second camera to capture an image of an area fed with a paving material by the left screw.
  • the left peripheral image LG that is a component of the peripheral image may be generated based on an image captured by the left camera 51L and the left local image SGL composing the local image may be generated based on an image captured by the left auxiliary camera 51U.
  • the image generating system SYS may include the right camera 51R that captures an image of an area to the right of the asphalt finisher 100 and the right local area fed with a paving material by the right screw.
  • the right peripheral image RG and the right local image SGR may be generated based on an image captured by the right camera 51R.
  • the right auxiliary camera 51V may be omitted.
  • the image generating system SYS may include the left camera 51L that captures an image of an area to the left of the asphalt finisher 100 and the left local area fed with a paving material by the left screw.
  • the left peripheral image LG and the left local image SGL may be generated based on an image captured by the left camera 51L.
  • the left auxiliary camera 51U may be omitted.
  • the local image may be displayed without overlapping the peripheral image as illustrated in FIGS. 4A through 4C , for example, and may be displayed over the peripheral image as illustrated in FIGS. 6 through 8 , for example.
  • the display device 52 may display the indicator BG that represents the state of enlargement or reduction of the local image SG. By looking at the indicator BG, the operator can instantaneously understand the state of enlargement or reduction of each image part of the local image SG.
  • An asphalt finisher that displays images captured by a front camera, a left camera, and a right camera side by side around a computer-generated graphic of a tractor has been known (see Patent Document 1).
  • the front camera is attached to the upper end of the front of the tractor to capture an image of the inside of a hopper in front of the tractor.
  • the left camera is attached to the upper end of the left side of the tractor to capture an image of a space to the left of the asphalt finisher.
  • the right camera is attached to the upper end of the right side of the tractor to capture an image of a space to the right of the asphalt finisher.
  • a space around the front end of hopper wings is the blind area of the hopper wings when viewed from the front camera, the left camera, and the right camera. Therefore, an operator of the asphalt finisher cannot understand the state of the space around the front end of the hopper wings by looking at the displayed image. The operator has to check the safety of a space that is invisible in an image directly with her/his eyes.
  • FIG. 10 is a side view of the asphalt finisher 100 that is an example of a road machine according to an embodiment of the invention.
  • FIG. 11 is a plan view of the asphalt finisher 100.
  • the asphalt finisher 100 mainly includes the tractor 1, the hopper 2, and the screed 3.
  • the direction of the hopper 2 as viewed from the tractor 1 is defined as a forward direction (the +X direction)
  • the direction of the screed 3 as viewed from the tractor 1 is defined as a rearward direction (the -X direction).
  • the tractor 1 is a vehicle for causing the asphalt finisher 100 to travel. According to this embodiment, the tractor 1 rotates rear wheels 5 using rear wheel traveling hydraulic motors and rotates front wheels 6 using front wheel traveling hydraulic motors to move the asphalt finisher 100.
  • the rear wheel traveling hydraulic motors and the front wheel traveling hydraulic motors are supplied with hydraulic oil from a hydraulic pump to rotate.
  • the rear wheels 5 and the front wheels 6 may be replaced with crawlers.
  • the tractor 1 includes a canopy 1C.
  • the canopy 1C is attached to the top of the tractor 1.
  • the controller 50 is a control device to control the asphalt finisher 100.
  • the controller 50 is composed of a microcomputer including a CPU, a volatile memory, and a nonvolatile memory, and is installed in the tractor 1.
  • the CPU executes programs stored in the nonvolatile memory to implement various functions of the controller 50.
  • the hopper 2 is a mechanism for receiving a paving material, and mainly includes hopper wings 20 and hopper cylinders 24. According to this embodiment, the hopper 2 is installed in front of the tractor 1 and receives a paving material in the hopper wings 20.
  • the hopper wings 20 include a left hopper wing 20L that can be opened and closed in the Y-axis directions (widthwise of the vehicle) by a left hopper cylinder 24L and a right hopper wing 20R that can be opened and closed in the Y-axis directions (widthwise of the vehicle) by a right hopper cylinder 24R.
  • the asphalt finisher 100 fully opens the hopper wings 20 to receive a paving material (for example, an asphalt mixture) from the bed of a dump truck.
  • FIG. 11 illustrates that the hopper wings 20 are fully open.
  • the hopper wings 20 are closed to gather the paving material near the inner wall of the hopper 2 to the center of the hopper 2 so that the conveyor CV in the center of the hopper 2 can continuously feed the paving material to the back of the tractor 1, that is, the paving material can be kept piled on the conveyor CV.
  • the screw SC has extension screws laterally coupled.
  • the screed 3 is a mechanism for spreading and smoothing a paving material.
  • the screed 3 includes a front screed 30 and rear screeds 31.
  • the screed 3 is a free floating screed towed by the tractor 1, and is coupled to the tractor 1 via leveling arms 3A.
  • the rear screeds 31 include the left rear screed 31L and the right rear screed 31R.
  • the left rear screed 31L extends and retracts widthwise of the vehicle using a left screed extendable and retractable cylinder 26L
  • the right rear screed 31R extends and retracts widthwise of the vehicle using a right screed extendable and retractable cylinder 26R.
  • the image capturing devices 51 are devices to capture images. According to this embodiment, the image capturing devices 51 are monocular cameras, and are connected to the controller 50 wirelessly or by wire. The controller 50 can generate an overhead view image by performing a viewpoint changing process on images captured by the image capturing devices 51.
  • the overhead view image is, for example, an image of a space around the asphalt finisher 100 as virtually viewed from substantially directly above.
  • the image capturing devices 51 may be stereo cameras.
  • the image capturing devices 51 include the front camera 51F, the left camera 51L, the right camera 51R, and a back camera 51B.
  • the back camera 51B may be omitted.
  • the front camera 51F captures an image of a space in front of the asphalt finisher 100.
  • the front camera 51F is attached to a hood forming the front of the tractor 1 so as to be able to capture an image of the inside of the hopper 2 that is a blind area from the viewpoint of the operator seated in the operator seat 1S (hereinafter, "operator seat viewpoint").
  • the front camera 51F may be attached to the front edge of the top plate of the canopy 1C.
  • a gray area Z1 indicates the imaging range of the front camera 51F.
  • the left camera 51L captures an image of a space to the left of the asphalt finisher 100.
  • the left camera 51L is attached to the end of a bar member BL extending in the +Y direction (leftward) from the left edge of the top plate of the canopy 1C so as to be able to capture an image of a space outside the left hopper wing 20L in the vehicle width direction that is a blind area from the operator seat viewpoint.
  • the left camera 51L may be attached to the end of the bar member BL extending in the +Y direction (leftward) from the right side of the tractor 1.
  • the left camera 51L is so attached as to protrude outward (leftward) in the vehicle width direction relative to the left end of the fully opened left hopper wing 20L.
  • a gray area Z2 indicates the imaging range of the left camera 51L.
  • the right camera 51R captures an image of a space to the right of the asphalt finisher 100.
  • the right camera 51R is attached to the end of a bar member BR extending in the -Y direction (rightward) from the right edge of the top plate of the canopy 1C so as to be able to capture an image of a space outside the right hopper wing 20R in the vehicle width direction that is a blind area from the operator seat viewpoint.
  • the right camera 51R may be attached to the end of the bar member BR extending in the -Y direction (rightward) from the right side of the tractor 1.
  • the right camera 51R is so attached as to protrude outward (rightward) in the vehicle width direction relative to the right end of the fully opened right hopper wing 20R.
  • a gray area Z3 indicates the imaging range of the right camera 51R.
  • the bar members BL and BR are desirably removable.
  • the bar members BL and BR may be extendable and retractable. This is for handling the case of transporting the asphalt finisher 100 in a trailer or the like.
  • the back camera 51B captures an image of a space behind the asphalt finisher 100.
  • the back camera 51B is attached to the rear edge of the top plate of the canopy 1C so as to be able to capture an image of a space behind the screed 3 that is a blind area from the operator seat viewpoint.
  • a gray area Z4 indicates the imaging range of the back camera 51B.
  • the imaging range of the front camera 1F and the imaging range of the left camera 1L may overlap.
  • the imaging range of the back camera 1B and the imaging range of the left camera 1L do not have to overlap. The same applies to the imaging range of the right camera 1R.
  • the controller 50 generates an overhead view image by changing the viewpoints of and synthesizing the respective captured images of the front camera 51F, the left camera 51L, the right camera 51R, and the back camera 51B.
  • the overhead view image includes an image of the internal space of the hopper 2, a space to the left of the left hopper wing 20L, a space to the right of the right hopper wing 20R, and a space behind the screed 3 as virtually viewed from substantially directly above and a computer graphics image (hereinafter, "model image") of the asphalt finisher 100.
  • the controller 50 may generate an overhead view image by changing the viewpoints of and synthesizing the respective captured images of the three cameras of the front camera 51F, the left camera 51L, and the right camera 51R. That is, the overhead view image may be generated without using the back camera 51B.
  • the display device 52 is a device to display various images.
  • the display device 52 is a liquid crystal display, and is connected to the controller 50 wirelessly or by wire.
  • the display device 52 can display an image captured by each of the image capturing devices 51, and is placed at such a position as to make it easy for an operator seated in the operator seat 1S to look at the display device 52.
  • the display device 52 may be placed where a rear controller is.
  • the controller 50 displays an image generated by performing a viewpoint changing process on images captured by the image capturing devices 51 on the display device 52.
  • FIG. 12 is a block diagram illustrating an example configuration of the display system GS.
  • the display system GS mainly includes the controller 50, the image capturing devices 51, the display device 52, an information obtaining device 53, and the storage device 54.
  • the display system GS generates an image for display (hereinafter, "output image") based on images captured by the image capturing devices 51 (hereinafter, "input images”) and displays the output image on the display device 52.
  • the information obtaining device 53 obtains information and outputs the obtained information to the controller 50.
  • the information obtaining device 53 includes at least one of, for example, a hopper cylinder stroke sensor, screed extendable and retractable cylinder stroke sensors, a steering angle sensor, a travel speed sensor, and a positioning sensor.
  • the hopper cylinder stroke sensor detects the stroke amount of the hopper cylinders 24.
  • the screed extendable and retractable cylinder stroke sensors detect the stroke amounts of screed extendable and retractable cylinders 26.
  • the steering angle sensor detects the steering angle of the front wheels 6.
  • the travel speed sensor detects the travel speed of the asphalt finisher 100.
  • the positioning sensor is, for example, a GNSS compass, and detects the position (latitude, longitude, and altitude) and the orientation of the asphalt finisher 100.
  • the storage device 54 is a device for storing various kinds of information. According to this example, the storage device 54 is a nonvolatile storage device that stores the input image-output image correspondence map 54a in such a manner as to allow reference to the input image-output image correspondence map 54a.
  • the input image-output image correspondence map 54a stores the correspondence between the coordinates in the input image planes and the coordinates in the output image plane.
  • the correspondence is preset based on various parameters of the image capturing devices 51, such as an optical center, a focal length, a CCD size, an optical axis direction vector, a camera horizontal direction vector, and a projection method, so that a viewpoint can be changed as desired.
  • the correspondence is so set as to prevent an apparent distortion or tilt/shift from appearing in the output image.
  • the controller 50 includes a viewpoint changing part 50a and an auxiliary line creating part 50b.
  • the viewpoint changing part 50a and the auxiliary line creating part 50b are composed of software, hardware, or firmware.
  • the viewpoint changing part 50a is a functional element to generate the output image.
  • the viewpoint changing part 50a refers to the input image-output image correspondence map 54a stored in the storage device 54 to correlate the coordinates in the input image planes in which input images captured by the image capturing devices 51 are positioned with the coordinates in the output image plane in which the overhead view image as the output image is positioned.
  • the viewpoint changing part 50a generates the output image by correlating the values (for example, luminance value, hue value, chromatic value, etc.) of pixels in the input images with the values of pixels in the output image.
  • the auxiliary line creating part 50b is a functional element to create auxiliary lines to be displayed over the output image. According to this embodiment, the auxiliary line creating part 50b creates auxiliary lines such that the auxiliary lines match the overhead view image generated by the viewpoint changing part 50a.
  • auxiliary lines include an auxiliary line indicating an expected pavement trajectory that is the expected trajectory of an end of the screed 3 and an auxiliary line indicating an expected travel trajectory that is the expected trajectory of a wheel.
  • the controller 50 refers to the input image-output image correspondence map 54a through the viewpoint changing part 50a. Then, the controller 50 obtains the values (for example, luminance value, hue value, chromatic value, etc.) of coordinates in the input image planes corresponding to the coordinates in the output image plane, and adopts the obtained values as the values of the corresponding coordinates in the output image plane.
  • the values for example, luminance value, hue value, chromatic value, etc.
  • the controller 50 determines whether the values of all the coordinates in the output image plane are correlated with the values of coordinates in the input image planes. In response to determining that the values of all the coordinates are not correlated, the controller 50 repeats the above-described process.
  • the controller 50 In response to determining that the values of all the coordinates are correlated, the controller 50 displays auxiliary lines indicating expected pavement trajectories, auxiliary lines indicating expected travel trajectories, etc., over the output image. Positions in the output image where auxiliary lines are to be superposed, which are preset according to this embodiment, may also be dynamically derived. Furthermore, the controller 50 may correlate the coordinates in the input image planes and the coordinates in the output image plane after displaying auxiliary lines.
  • FIGS. 13A and 13B are diagrams illustrating an example of the overhead view image.
  • FIG. 13A illustrates an example of the overhead view image displayed on the display device 52.
  • FIG. 13B illustrates the segments of input images used for generating the overhead view image of FIG. 13A .
  • the overhead view image of FIG. 13A is an image of a space around the asphalt finisher 100 as virtually viewed from substantially directly above.
  • the overhead view image of FIG. 13A mainly includes an image G1 (see the hatched area) generated by the viewpoint changing part 50a and a model image CG1.
  • the model image CG1 is an image representing the asphalt finisher 100, and includes a model image CGa of the hopper 2, a model image CGb of the tractor 1, and a model image CGc of the screed 3.
  • the model image CGa of the hopper 2 includes a model image WL of the left hopper wing 20L and a model image WR of the right hopper wing 20R.
  • the model images WL and WR change in shape according to the output of the hopper cylinder stroke sensor.
  • FIG. 13A illustrates the model image CGa when each of the left hopper wing 20L and the right hopper wing 20R is fully open.
  • Part of the overhead view image (an image of the inside of the hopper 2 as virtually viewed from substantially directly above) is placed in the hatched area between the model image WL and the model image WR.
  • the model image CGa of the hopper 2 may be omitted. In this case, part of the overhead view image based on an image captured by the front camera 1F is placed.
  • the model image CGc of the screed 3 includes a model image SL of the left rear screed 31L and a model image SR of the right rear screed 31R.
  • the model images SL and SR change in shape according to the outputs of the screed extendable and retractable cylinder stroke sensors.
  • FIG. 13A illustrates the model image CGc when each of the left rear screed 31L and the right rear screed 31R extends most.
  • the model image CGc of the screed 3 may be omitted. In this case, part of the overhead view image based on an image captured by the back camera 1B is placed.
  • the image G1 is an image generated using the respective captured input images of the four image capturing devices 51.
  • the image G1 includes an image Ga of a worker in front and to the left of the asphalt finisher 100 and an image Gb of a maintenance hole cover in front and to the right of the asphalt finisher 100.
  • the controller 50 generates the image G1 by synthesizing a front image R1, a left image R2, a right image R3, and a rear image R4.
  • the worker image Ga is included in the left image R2, and the maintenance hole cover image Gb is included in the right image R3.
  • the front image R1 is an image generated based on an input image captured by the front camera 51F.
  • the front image R1 includes an image showing the state of the inside of the hopper 2 as viewed down from the tractor 1 side.
  • the controller 50 generates the front image R1 by clipping and performing a viewpoint changing process on part of the input image captured by the front camera 51F.
  • the front image R1 is placed between and on the upper side of the model image WL and the model image WR.
  • the front image R1 may change in shape according as the model images WL and WR change in shape.
  • the left image R2 is an image generated based on an input image captured by the left camera 51L.
  • the left image R2 includes an image of a space on the outer side (to the left) of the left hopper wing 20L in the vehicle width direction.
  • the controller 50 generates the left image R2 by clipping and performing a viewpoint changing process by the viewpoint changing part 50a on part of the input image captured by the left camera 51L.
  • the left image R2 is placed to the left of the model image CG1.
  • the left image R2 may change in shape according as the model images WL and SL change in shape.
  • the right image R3 is an image generated based on an input image captured by the right camera 51R.
  • the right image R3 includes an image of a space on the outer side (to the right) of the right hopper wing 20R in the vehicle width direction.
  • the controller 50 generates the right image R3 by clipping and performing a viewpoint changing process by the viewpoint changing part 50a on part of the input image captured by the right camera 51R.
  • the right image R3 is placed to the right of the model image CG1.
  • the right image R3 may change in shape according as the model images WR and SR change in shape.
  • the rear image R4 is an image generated based on an input image captured by the back camera 51B.
  • the rear image R4 includes an image showing the state of the screed 3 as viewed down from the tractor 1 side.
  • the controller 50 generates the rear image R4 by clipping and performing a viewpoint changing process on part of the input image captured by the back camera 51B.
  • the rear image R4 is placed on the lower side of the model image CGc of the screed 3.
  • the rear image R4 may change in shape according as the model images SL and SR change in shape.
  • the correspondence between the coordinates in the respective input image planes of the four cameras and the coordinates in the output image plane is stored in the input image-output image correspondence map 54a with effects according to the viewpoint changing process being incorporated in advance. Therefore, the controller 50 can correlate coordinates in the input image planes with coordinates in the output image plane by only referring to the input image-output image correspondence map 54a. As a result, the controller 50 can generate and display an output image at a relatively low computational load.
  • each of the front image R1, the left image R2, the right image R3, and the rear image R4 is generated based on an input image captured by a corresponding single camera.
  • the invention is not limited to this configuration.
  • each of the front image R1, the left image R2, the right image R3, and the rear image R4 may be generated based on input images captured by two or more cameras.
  • the asphalt finisher 100 includes the multiple image capturing devices 51 attached to the tractor 1, the controller 50 that generates an overhead view image by changing the viewpoints of and synthesizing the respective captured images of the image capturing devices 51, and the display device 52 that displays the overhead view image on a screen.
  • the overhead view image is configured to include an image of a space around the asphalt finisher 100 as virtually viewed from substantially directly above. Therefore, it is possible to further reduce a blind area that is an invisible area in the output image.
  • the asphalt finisher 100 can improve visibility, safety, operability and work efficiency. Specifically, the asphalt finisher 100 can cause the operator to intuitively understand the amount of a paving material remaining in the hopper 2, the position of a feature (for example, a maintenance hole) in a road surface to be paved, etc. Therefore, the operator can look at the overhead view image and check the position of a feature or a worker, and thereafter perform various operations such as opening or closing the hopper wings 20.
  • a feature for example, a maintenance hole
  • FIGS. 14A and 14B illustrate examples where auxiliary lines are displayed over the image G1 generated by the viewpoint changing part 50a.
  • FIG. 14A illustrates an overhead view image of the asphalt finisher 100 that is traveling straight.
  • FIG. 14B illustrates an overhead view image of the asphalt finisher 100 that is turning right.
  • the auxiliary line creating part 50b creates auxiliary lines L1 and L2 based on the respective outputs of the steering angle sensor and the travel speed sensor.
  • the auxiliary line L1 is the expected travel trajectory of a left rear wheel 5L
  • the auxiliary line L2 is the expected travel trajectory of a right rear wheel 5R.
  • the auxiliary lines L1 and L2 are derived based on a current steering angle and travel speed, but may be derived based solely on the steering angle.
  • the auxiliary lines L1 and L2 represent, for example, travel trajectories during a period from a current point of time until a predetermined time (for example, several tens of seconds) passes.
  • the auxiliary line creating part 50b creates auxiliary lines L3 and L4, additionally referring to the outputs of the screed extendable and retractable cylinder stroke sensors.
  • the auxiliary line L3 is the expected trajectory of the left end of the left rear screed 31L
  • the auxiliary line L4 is the expected trajectory of the right end of the right rear screed 31R.
  • the auxiliary lines L3 and L4 are derived based on a current steering angle, travel speed, and stroke amounts of the screed extendable and retractable cylinders 26.
  • the auxiliary lines L3 and L4 represent trajectories during a period from a current point of time until a predetermined time (for example, several tens of seconds) passes.
  • the auxiliary line creating part 50b creates auxiliary lines L5 and L6 based on road design data and the output of the positioning sensor.
  • the road design data are data related to a road that is a construction target, and is prestored in a nonvolatile storage device, for example.
  • the road design data include, for example, data on the position of a feature in a road surface to be paved.
  • the auxiliary line L5 represents the left edge of the road that is a construction target, and the auxiliary line L6 represents the right edge of the road that is a construction target.
  • the controller 50 displays an image of a feature such as a maintenance hole over the overhead view image based on the road design data and the output of the positioning sensor. According to the illustration of FIGS. 14A and 14B , the controller 50 displays a model image CG2 of a maintenance hole cover over the overhead view image.
  • the operator of the asphalt finisher 100 can determine whether the current steering angle, travel speed, amounts of extension or retraction of the rear screeds 31, etc., of the asphalt finisher 100 are appropriate. For example, by looking at the overview head image of FIG. 14A , the operator can be aware that if the asphalt finisher 100 is kept traveling straight, the right rear wheel 5R will run over the maintenance hole cover and the road will not be paved as designed. In contrast, by looking at the overhead view image of FIG. 14B , the operator can be aware that if a current steering condition (condition in which a steering wheel is turned to the right) and travel speed are maintained, the right rear wheel 5R can avoid running over the maintenance hole cover and the road will be paved as designed.
  • a current steering condition condition in which a steering wheel is turned to the right
  • the controller 50 which displays the expected travel trajectories of the rear wheels 5 according to the illustration of FIGS. 14A and 14B , may display the expected travel trajectories of the front wheels 6 or display the respective expected travel trajectories of the rear wheels 5 and the front wheels 6.
  • the controller 50 can achieve the additional effect of being able to present the movement of the asphalt finisher 100 during a period from a current point of time until a predetermined time passes to the operator in advance.
  • the image part may be enlarged and reduced in size only laterally or may be enlarged and reduced in size vertically and laterally.
  • the indicator BG may be a laterally elongated bar-shaped indicator.
  • the indicator BG may be a vertically and laterally elongated matrix-shaped indicator.
  • the asphalt finisher 100 may be a guss asphalt finisher using a guss asphalt mixture.
  • the image generating system SYS may be installed in a guss asphalt finisher using a guss asphalt mixture.
  • auxiliary line creating part 51 ... image capturing device 51B ... back camera 51F ... front camera 51L ... left camera 51R ... right camera 51U ... left auxiliary camera 51V ... right auxiliary camera 52 ... display device 53 ... information obtaining device 54 ... storage device 54a ... input image-output image correspondence map 55 ... input device 65 ... operation panel 70 ... retaining plate 71 ... side plate 72 ... mold board 100 ... asphalt finisher BL, BR ... bar member CV ... conveyor SC ... screw SYS ... image generating system
EP18843681.0A 2017-08-08 2018-07-31 Strassenmaschine Active EP3666977B1 (de)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
JP2017153668 2017-08-08
JP2017164687 2017-08-29
PCT/JP2018/028750 WO2019031318A1 (ja) 2017-08-08 2018-07-31 道路機械

Publications (3)

Publication Number Publication Date
EP3666977A1 true EP3666977A1 (de) 2020-06-17
EP3666977A4 EP3666977A4 (de) 2021-01-27
EP3666977B1 EP3666977B1 (de) 2023-12-13

Family

ID=65271383

Family Applications (1)

Application Number Title Priority Date Filing Date
EP18843681.0A Active EP3666977B1 (de) 2017-08-08 2018-07-31 Strassenmaschine

Country Status (4)

Country Link
EP (1) EP3666977B1 (de)
JP (3) JP7146767B2 (de)
CN (1) CN111032958B (de)
WO (1) WO2019031318A1 (de)

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP3779047A4 (de) * 2018-03-30 2021-04-21 Sumitomo (S.H.I.) Construction Machinery Co., Ltd. Strassenmaschine
WO2022100838A1 (de) * 2020-11-12 2022-05-19 Moba Mobile Automation Ag Bediensystem für eine baumaschine
CN114763701A (zh) * 2021-01-14 2022-07-19 现代斗山英维高株式会社 工程机械的控制系统及方法
US11724652B2 (en) 2019-06-11 2023-08-15 Kubota Corporation Protection mechanism for working vehicle, and working vehicle including the same
US11884141B2 (en) 2019-06-11 2024-01-30 Kubota Corporation Protection mechanism for working vehicle, and working vehicle including the same

Families Citing this family (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP7317946B2 (ja) * 2019-03-25 2023-07-31 住友建機株式会社 アスファルトフィニッシャ
CN113597489A (zh) * 2019-03-29 2021-11-02 住友建机株式会社 沥青滚平机
CN113924395A (zh) * 2019-03-29 2022-01-11 住友建机株式会社 沥青滚平机
JP7187389B2 (ja) * 2019-06-11 2022-12-12 株式会社クボタ 作業車両の保護機構、及びこれを備える作業車両
JPWO2021193351A1 (de) * 2020-03-26 2021-09-30
WO2021193381A1 (ja) * 2020-03-27 2021-09-30 住友建機株式会社 アスファルトフィニッシャ及び機械学習装置
CN111809481B (zh) * 2020-07-21 2022-04-19 三一汽车制造有限公司 摊铺机输料车引导系统、摊铺机和摊铺机输料车引导方法

Family Cites Families (26)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPS6029941B2 (ja) 1979-06-27 1985-07-13 株式会社日立製作所 複合型の電子写真板
JPH077368Y2 (ja) * 1990-11-13 1995-02-22 株式会社新潟鐵工所 アスファルトフィニッシャにおける施工監視装置
JP2505210Y2 (ja) * 1993-04-09 1996-07-24 建設省東北地方建設局長 舗装作業車の自動操向装置
JPH089849B2 (ja) * 1993-04-09 1996-01-31 建設省東北地方建設局長 アスファルトフィニッシャの自動制御装置
JPH11217853A (ja) * 1998-01-30 1999-08-10 Komatsu Ltd 建設機械の後方監視装置およびその後方監視方法
JP4907883B2 (ja) * 2005-03-09 2012-04-04 株式会社東芝 車両周辺画像表示装置および車両周辺画像表示方法
JP4776491B2 (ja) * 2006-10-06 2011-09-21 日立建機株式会社 作業機械の周囲監視装置
JP4705611B2 (ja) * 2007-07-10 2011-06-22 住友建機株式会社 舗装機械の運転者支援装置
JP5269026B2 (ja) * 2010-09-29 2013-08-21 日立建機株式会社 作業機械の周囲監視装置
JP5809988B2 (ja) * 2012-01-10 2015-11-11 日立建機株式会社 作業機械の走行支援装置
WO2013136374A1 (ja) 2012-03-16 2013-09-19 三菱電機株式会社 運転支援装置
JP5917371B2 (ja) * 2012-11-08 2016-05-11 住友重機械工業株式会社 舗装機械用画像生成装置及び舗装機械用操作支援システム
JP6073182B2 (ja) * 2013-04-26 2017-02-01 住友重機械工業株式会社 舗装機械用画像生成装置及び舗装機械用操作支援システム
JP6029941B2 (ja) * 2012-11-08 2016-11-24 住友重機械工業株式会社 舗装機械用画像生成装置
JP6279856B2 (ja) * 2013-08-27 2018-02-14 住友建機株式会社 ショベル用表示装置
JP2015104375A (ja) * 2013-12-02 2015-06-08 ヤンマー株式会社 コンバイン
JP6095592B2 (ja) * 2014-02-17 2017-03-15 日立建機株式会社 油圧ショベルの監視画像表示装置
CN104278616A (zh) * 2014-09-28 2015-01-14 广东惠利普路桥信息工程有限公司 无人驾驶的摊铺机
JP2016139914A (ja) * 2015-01-27 2016-08-04 パイオニア株式会社 表示装置、携帯端末及び制御方法
DE102015002692A1 (de) * 2015-03-04 2016-09-08 Dynapac Gmbh Straßenbaumaschine und Verfahren zum Betreiben einer selbstfahrenden Straßenbaumaschine
JPWO2016174754A1 (ja) 2015-04-28 2018-02-15 株式会社小松製作所 作業機械の周辺監視装置及び作業機械の周辺監視方法
SE539312C2 (en) * 2015-06-10 2017-06-27 Conny Andersson Med Firma Ca Konsult A method of determining the quality of a newly produced asphalt pavement
JP6913630B2 (ja) * 2015-08-24 2021-08-04 株式会社小松製作所 ホイールローダの制御システム、その制御方法およびホイールローダの制御方法
JP2017089325A (ja) * 2015-11-16 2017-05-25 住友建機株式会社 アスファルトフィニッシャ
JP6677534B2 (ja) 2016-03-01 2020-04-08 株式会社三共 スロットマシン
JP2017164687A (ja) 2016-03-16 2017-09-21 中間貯蔵・環境安全事業株式会社 Ｐｃｂ汚染機器解体方法

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP3779047A4 (de) * 2018-03-30 2021-04-21 Sumitomo (S.H.I.) Construction Machinery Co., Ltd. Strassenmaschine
US11724652B2 (en) 2019-06-11 2023-08-15 Kubota Corporation Protection mechanism for working vehicle, and working vehicle including the same
US11884141B2 (en) 2019-06-11 2024-01-30 Kubota Corporation Protection mechanism for working vehicle, and working vehicle including the same
WO2022100838A1 (de) * 2020-11-12 2022-05-19 Moba Mobile Automation Ag Bediensystem für eine baumaschine
CN114763701A (zh) * 2021-01-14 2022-07-19 现代斗山英维高株式会社 工程机械的控制系统及方法
EP4029998A1 (de) * 2021-01-14 2022-07-20 Hyundai Doosan Infracore Co., Ltd. System und verfahren zur steuerung einer baumaschine

Also Published As

Publication number Publication date
JP2022111290A (ja) 2022-07-29
EP3666977A4 (de) 2021-01-27
WO2019031318A1 (ja) 2019-02-14
CN111032958B (zh) 2023-03-10
CN111032958A (zh) 2020-04-17
JP2024028606A (ja) 2024-03-04
EP3666977B1 (de) 2023-12-13
JP7146767B2 (ja) 2022-10-04
JPWO2019031318A1 (ja) 2020-09-24

Similar Documents

Publication Publication Date Title
EP3666977B1 (de) Strassenmaschine
US10650251B2 (en) Monitoring image display device of industrial machine
EP2918726B1 (de) Strassenpflasterungsmaschine mit bilderzeugungsvorrichtung
EP2918725B1 (de) Bilderzeugungsvorrichtung für eine strassenpflasterungsmaschine und betriebshilfssystem für die strassenpflasterungsmaschine
JP5888956B2 (ja) ショベル及び該ショベルの周囲画像表示方法
AU2013293921B2 (en) Environment monitoring device for operating machinery
JP7445591B2 (ja) 道路機械
US20150009329A1 (en) Device for monitoring surroundings of machinery
US10621743B2 (en) Processing-target image creating device, processing-target image creating method, and operation assisting system
JP2019007307A (ja) アスファルトフィニッシャ
JP6746303B2 (ja) ショベル
JP7178334B2 (ja) ショベル及びショベルの表示装置
JP7009327B2 (ja) ショベル

Legal Events

Date Code Title Description
STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE INTERNATIONAL PUBLICATION HAS BEEN MADE

PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE

17P Request for examination filed

Effective date: 20200210

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

AX Request for extension of the european patent

Extension state: BA ME

DAV Request for validation of the european patent (deleted)
DAX Request for extension of the european patent (deleted)
STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: EXAMINATION IS IN PROGRESS

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: EXAMINATION IS IN PROGRESS

A4 Supplementary search report drawn up and despatched

Effective date: 20210112

RIC1 Information provided on ipc code assigned before grant

Ipc: E01C 19/48 20060101AFI20201221BHEP

17Q First examination report despatched

Effective date: 20210122

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: GRANT OF PATENT IS INTENDED

INTG Intention to grant announced

Effective date: 20230705

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE PATENT HAS BEEN GRANTED

RIN1 Information on inventor provided before grant (corrected)

Inventor name: HAGIWARA, KAZUAKI

Inventor name: BABA, NOBUYUKI

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 602018062708

Country of ref document: DE

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240314

REG Reference to a national code

Ref country code: LT

Ref legal event code: MG9D