JP2024028606A - road machinery - Google Patents

Abstract

[Problem] To provide a road machine that can pave roads as designed. An asphalt finisher (100) includes an imaging device (51), a controller (50), a display device (52), and an information acquisition device (53). The controller 50 generates an overhead image of the space around the asphalt finisher 100 based on the image captured by the imaging device 51, creates an expected travel trajectory of the wheels based on the information acquired by the information acquisition device 53, and generates the overhead image. , the expected travel trajectory, and images of features on the road are displayed on the display device 52. [Selection diagram] Figure 12

Description

The present invention relates to a road machine equipped with a screw for axially feeding paving material.

BACKGROUND ART Conventionally, there has been known an image generation device that generates an image that shows an asphalt finisher and its surroundings when viewed from above and presents the generated image to the driver of the asphalt finisher (see Patent Document 1). This image generation device generates and displays an image showing the entire surroundings of the asphalt finisher, thereby allowing the driver to intuitively recognize the positional relationship between the asphalt finisher and surrounding objects.

Patent No. 6029941

However, in the above configuration, the front camera, left camera, and right camera are attached to the tractor.

In view of the above, it is desired to provide a road machine that can pave roads as designed.

A road machine according to an embodiment of the present invention includes a tractor, a hopper installed on the front side of the tractor and receiving paving material in a hopper wing, and a conveyor that feeds the paving material in the hopper to the rear side of the tractor. a screw for spreading the paving material fed by the conveyor on the rear side of the tractor; a screed for spreading the paving material spread by the screw on the rear side of the screw; The vehicle includes a plurality of attached imaging devices, a control device, a display device, and an information acquisition device, and the control device generates an overhead image of the space around the road machine based on the image captured by the imaging device. A predicted travel trajectory of the wheels is created based on the information acquired by the information acquisition device, and the overhead image, the predicted travel trajectory, and an image of a feature on the road are displayed on the display device.

By means of the above-mentioned means, a road machine is provided which can pave roads as designed.

FIG. 1 is a side view of an asphalt finisher according to an embodiment of the present invention. FIG. 1 is a top view of an asphalt finisher according to an embodiment of the present invention. FIG. 2 is a rear view of an asphalt finisher according to an embodiment of the present invention. 1A is a diagram illustrating a configuration example of an image generation system installed in the asphalt finisher of FIG. 1A. FIG. This is a display example of the first output image. This is a display example of the second output image. This is a display example of the second output image. This is a display example of the second output image. 7 is a flowchart of output image generation processing. 3 is a flowchart of output image switching processing. It is another display example of a 2nd output image. This is yet another display example of the second output image. This is yet another display example of the second output image. FIG. 1 is a side view of an asphalt finisher according to an embodiment of the present invention. 11 is a top view of the asphalt finisher of FIG. 10. FIG. 11 is a block diagram showing a configuration example of a display system installed in the asphalt finisher of FIG. 10. FIG. 13 is a diagram showing an example of an image generated by the display system of FIG. 12. FIG. FIG. 13B is a diagram illustrating the partitioning of the input image used to generate the image of FIG. 13A. 13 is a diagram showing another example of an image generated by the display system of FIG. 12. FIG. 13 is a diagram showing yet another example of an image generated by the display system of FIG. 12. FIG.

DESCRIPTION OF THE PREFERRED EMBODIMENTS The best mode for carrying out the present invention will be described below with reference to the drawings.

1A to 1C show a configuration example of an asphalt finisher 100 as a road machine according to an embodiment of the present invention, in which FIG. 1A shows a side view, FIG. 1B shows a top view, and FIG. 1C shows a rear view. .

The asphalt finisher 100 mainly includes a tractor 1, a hopper 2, and a screed 3.

The tractor 1 is a device for driving the asphalt finisher 100 and pulls the screed 3. In this embodiment, the tractor 1 uses a travel hydraulic motor to rotate two or four wheels to move the asphalt finisher 100. The travel hydraulic motor rotates by receiving hydraulic fluid from a hydraulic pump driven by a prime mover such as a diesel engine. A driver's seat 1S and an operation panel 65 are arranged at the upper part of the tractor 1.

Imaging devices 51 (right camera 51R, left camera 51L, front camera 51F, right auxiliary camera 51V, left auxiliary camera 51U) are attached to the tractor 1 on the right side, left side, and front. A display device 52 is installed at a position that is easily visible to the driver seated in the driver's seat 1S. In this embodiment, the direction of the hopper 2 as seen from the tractor 1 is the front (+X direction), and the direction of the screed 3 as seen from the tractor 1 is the rear (-X direction). The +Y direction corresponds to the left direction, and the -Y direction corresponds to the right direction.

The hopper 2, which is an example of a working device, is a mechanism for receiving paving material (for example, an asphalt mixture). The working device is a device that feeds the paving material in front of the screed 3. In this embodiment, it is configured to be openable and closable in the vehicle width direction using a hydraulic cylinder. The asphalt finisher 100 normally receives paving material from the loading platform of a dump truck with the hopper 2 fully open. When the amount of paving material in the hopper 2 decreases, the hopper 2 is closed and the paving material that was near the inner wall of the hopper 2 is collected in the center of the hopper 2, and the conveyor CV, which is an example of a work device, screeds the paving material. 3.

The screed 3 is a mechanism for leveling the paving material. In this embodiment, a hydraulic cylinder is configured to be able to move up and down in the vertical direction and expand and contract in the vehicle width direction. The width of the screed 3 is larger than the width of the tractor 1 when extended in the vehicle width direction. In this embodiment, the screed 3 includes a front screed 30, a left rear screed 31L, and a right rear screed 31R. The left rear screed 31L and the right rear screed 31R are configured to be able to expand and contract in the vehicle width direction (Y-axis direction). The left rear screed 31L and the right rear screed 31R, which are expandable and retractable in the vehicle width direction, are arranged offset from each other in the traveling direction (X-axis direction). Therefore, it can have a longer width (length in the vehicle width direction) than when it is not offset, can be extended longer in the vehicle width direction, and can construct a wider new pavement.

FIG. 2 schematically shows a configuration example of the image generation system SYS installed in the asphalt finisher 100 of FIG. 1A. The image generation system SYS generates an output image based on an input image captured by an imaging device 51 mounted on the asphalt finisher 100, for example. In this embodiment, the image generation system SYS mainly includes a controller 50, an imaging device 51, a display device 52, a storage device 54, and an input device 55.

The controller 50 is, for example, a computer including a CPU, volatile memory, nonvolatile memory, and the like. For example, the controller 50 causes the CPU to execute a program corresponding to each of the output image generation section 50A and the highlighted display section 50B, and realizes the functions corresponding to each of the output image generation section 50A and the highlighted display section 50B.

The imaging device 51 is a device that acquires an input image for generating an output image. In this embodiment, the camera is equipped with an image sensor such as a CCD or CMOS. The imaging device 51 is attached to the tractor 1 so as to be able to image the blind spot of a driver seated in the driver's seat 1S, for example. The blind spot includes, for example, the internal space of the hopper 2 (particularly the part near the tractor 1), the space outside the front end of the hopper 2, the space near the side of the asphalt finisher 100 near the road surface, and the like.

The imaging device 51 may be attached to a position other than the right side, left side, and front of the tractor 1 (for example, at the rear). The imaging device 51 may be equipped with a wide-angle lens or a fisheye lens. The imaging device 51 may be attached to the hopper 2 or the screed 3.

In this embodiment, the imaging device 51 includes a front camera 51F, a left camera 51L, a right camera 51R, a left auxiliary camera 51U, and a right auxiliary camera 51V. As shown in FIGS. 1A and 1B, the front camera 51F is attached to the upper end of the front part of the tractor 1, and its optical axis 51FX extends forward in the traveling direction, and forms an angle α with the road surface in a side view. It is installed to do so. As shown in FIGS. 1A to 1C, the left camera 51L is attached to the upper end of the left side of the tractor 1, and its optical axis 51LX forms an angle β with the left side surface of the tractor 1 in a top view. , and the road surface to form an angle γ when viewed from the rear. The right camera 51R is installed in the same way as the left camera 51L, with the left and right sides reversed. As shown in FIGS. 1A to 1C, the left auxiliary camera 51U is attached to the upper end of the left side of the tractor 1, and its optical axis 51UX forms an angle δ with the left side surface of the tractor 1 when viewed from above, Moreover, it is attached so as to form an angle ε with the road surface when viewed from the rear. The right auxiliary camera 51V is installed in the same manner as the left auxiliary camera 51U, with the left and right sides reversed. An area 51FA surrounded by a dashed line in FIG. 1B shows the imaging range of the front camera 51F, an area 51LA surrounded by a dashed-dotted line shows the imaging range of the left camera 51L, and an area 51RA surrounded by a dashed-dotted line shows the imaging range of the right camera 51R. shows the imaging range of An area 51UA surrounded by a two-dot chain line indicates an imaging range of the left auxiliary camera 51U, and an area 51VA surrounded by a two-dot chain line indicates an imaging range of the right auxiliary camera 51V.

The left camera 51L and the left auxiliary camera 51U are attached to the tractor 1 such that an area 51UA indicating the imaging range of the left auxiliary camera 51U is completely included in an area 51LA indicating the imaging range of the left camera 51L. However, it may be attached to the tractor 1 so that the area 51LA and the area 51UA partially overlap, that is, the area 51UA protrudes from the area 51LA. Similarly, the right camera 51R and the right auxiliary camera 51V are attached to the tractor 1 such that an area 51VA indicating the imaging range of the right auxiliary camera 51V is completely included in an area 51RA indicating the imaging range of the right camera 51R. . However, it may be attached to the tractor 1 so that the area 51RA and the area 51VA partially overlap, that is, the area 51VA protrudes from the area 51RA. Moreover, the left auxiliary camera 51U and the right auxiliary camera 51V may be omitted.

The imaging device 51 is attached to the asphalt finisher 100 via, for example, a bracket, a stay, a bar, or the like. In this embodiment, the imaging device 51 is attached to the tractor 1 via an attachment stay. However, the imaging device 51 may be directly attached to the tractor 1 without using an attachment stay, or may be embedded in the tractor 1.

In this embodiment, the imaging device 51 outputs the acquired input image to the controller 50. When acquiring an input image using a fisheye lens or a wide-angle lens, the imaging device 51 outputs to the controller 50 a corrected input image in which apparent distortion and tilt caused by using these lenses are corrected. It's okay. Alternatively, the input image without any apparent distortion or tilt correction may be output to the controller 50 as is. In this case, the apparent distortion and tilt are corrected by the controller 50.

In this way, the imaging device 51 is arranged so that its imaging range includes a plurality of blind spots on the left and right sides of the asphalt finisher 100 and on the inside and outside of the hopper 2.

The input device 55 is a device that allows the driver to input various information to the image generation system SYS, and is, for example, a touch panel, a button, a switch, or the like. In this embodiment, the input device 55 includes a display changeover switch and a screw dial.

The display changeover switch is a switch for changing over the configuration of the output image displayed on the display device 52. The screw dial is a dial for adjusting the rotational speed of a screw SC, which is an example of a working device.

The storage device 54 is a device for storing various information. In this embodiment, the storage device 54 is a non-volatile storage device and is integrated into the controller 50. However, the storage device 54 may be placed outside the controller 50 as a separate structure from the controller 50.

The display device 52 is a device for displaying various information. In this embodiment, it is a liquid crystal display installed on the operation panel 65, and displays various images output by the controller 50.

The output image generation unit 50A is a functional element for generating an output image, and is configured of, for example, software, hardware, or a combination thereof. In this embodiment, the output image generation unit 50A refers to the input image/output image correspondence map 54a stored in the storage device 54, and determines the coordinates on the input image plane where the input image captured by the imaging device 51 is located; The coordinates on the output image plane where the output image is located are associated with each other. Then, the output image generation unit 50A generates an output image by associating the value of each pixel in the output image (for example, brightness value, hue value, saturation value, etc.) with the value of each pixel in the input image. .

The input image/output image correspondence map 54a stores the correspondence relationship between the coordinates on the input image plane and the coordinates on the output image plane in a referable manner. The correspondence relationship is set in advance based on various parameters such as, for example, the optical center of the imaging device 51, the focal length, the CCD size, the optical axis direction vector, the camera horizontal direction vector, and the projection method. If the input image includes apparent distortion or tilt, the correspondence relationship may be set so that the apparent distortion or tilt does not appear in the output image. In this case, a group of coordinates forming a non-rectangular area on the input image plane is associated with a group of coordinates forming a rectangular area on the output image plane. The correspondence relationship is such that if the apparent distortion or tilt in the input image has already been corrected when the input image is acquired, the coordinates that make up a rectangular area on the input image plane will directly form the rectangular area on the output image plane. may be set to correspond to a coordinate group.

The highlighting section 50B is a functional element for switching the content of the output image displayed on the display device 52, and is configured of, for example, software, hardware, or a combination thereof. In this embodiment, the highlighting section 50B switches the output image displayed on the display device 52 between the first output image and the second output image when the display changeover switch serving as the highlighting switch is pressed. The first output image is switched to the second output image when the screw dial is operated, and then the second output image is switched to the first output image when the screw dial is not operated (non-operation time) reaches a predetermined time. You may switch to Similarly, when the highlighting switch is operated, the first output image is switched to the second output image, and after that, when the highlighting switch is not operated (non-operating time) reaches a predetermined time, the second output image is changed. may be switched to the first output image. The non-operation time is counted using a timer function of the controller 50, for example.

The first output image includes the surrounding image and does not include the local image. The second output image includes a surrounding image and a local image. The surrounding image is an image showing the surroundings of the asphalt finisher 100. The local image is an image showing a predetermined local area regarding the asphalt finisher 100, for example, an image showing an area where paving material is supplied (spread out) by the screw SC. The area to which the paving material is supplied by the screw SC is, for example, the area in front of the screed 3, and includes the retaining plate 70 (see FIG. 1B), the side plate 71 (see FIG. 1B), and the mold. This is the area surrounded by the board 72 (see FIG. 1B). By viewing this local image, the driver can view the local area while sitting in the driver's seat 1S without moving around on the tractor 1 or twisting his/her body in the driver's seat 1S to look into the local area. You can check the amount of paving material carried. In addition, the surrounding situation and the amount of paving material held in a local area can be confirmed almost simultaneously without having to move the line of sight significantly. In this way, the asphalt finisher 100 equipped with the image generation system SYS can reduce the driver's fatigue due to the work of checking the amount of cargo carried. As a result, safety regarding the asphalt finisher 100 can be improved. The local image may be an image showing the area inside the hopper 2.

Further, for example, when displaying the second output image on the display device 52, the highlighting section 50B causes the display device 52 to highlight the local image so that the driver can distinguish between the surrounding image and the local image. For example, the local image is displayed in a display frame that is different from the display frame that surrounds the surrounding images. In this case, the local image may be displayed on top of the surrounding image so as to overlap a part of the surrounding image, or may be displayed at a different position from the surrounding image so as not to overlap with the surrounding image. Alternatively, an image portion of the surrounding image that corresponds to a local area may be displayed in an enlarged manner. In this case, at least a portion of other image portions of the surrounding image may be displayed in a reduced size, or the display thereof may be omitted. Furthermore, when an image portion of the surrounding image that corresponds to a local area is displayed in an enlarged manner, display of the local image using another display frame may be omitted.

Next, with reference to FIG. 3, the first output image generated using the respective input images of the left camera 51L, right camera 51R, and front camera 51F will be described. FIG. 3 is a display example of the first output image displayed on the display device 52.

The first output image mainly includes a hopper image HG, a left surrounding image LG, a right surrounding image RG, and an illustration image 1CG. The hopper image HG, the left surrounding image LG, and the right surrounding image RG constitute a surrounding image. The image generation system SYS generates a hopper image HG, a left surrounding image LG, a right surrounding image RG, and an illustration image so that the driver can recognize that the front of the asphalt finisher 100 corresponds to the upper part of the screen of the display device 52. 1CG is displayed in a predetermined size at a predetermined position on the first output image. By presenting the first output image as an overhead image that shows the asphalt finisher 100 and its surroundings when viewed from above to the driver, the positional relationship between the asphalt finisher 100 and surrounding objects can be determined. This is to allow the driver to intuitively recognize this.

Hopper image HG is generated based on the input image of front camera 51F. In this embodiment, the hopper image HG is an image that shows the inside of the hopper 2 as seen when looking down on the hopper 2 from the tractor 1, and is generated by cutting out a part of the input image of the front camera 51F. , located at the upper center of the first output image.

The left surrounding image LG is generated based on the input image of the left camera 51L. In this embodiment, the left surrounding image LG is an image that shows the left surrounding area as seen when looking down from the tractor 1 on the left surrounding area on the left side of the asphalt finisher 100 in the direction of movement. Specifically, the left surrounding image LG is generated by cutting out a part of the input image of the left camera 51L, performing distortion correction, and further performing image rotation processing, and is placed at the left end of the first output image. Ru. Further, the left surrounding image LG includes an image of the left end of the screed 3 and an image of the left end of the hopper 2.

The right surrounding image RG is generated based on the input image of the right camera 51R. In the present embodiment, the right surrounding image RG is an image that shows the right surrounding area as seen when looking down from the tractor 1 on the right surrounding area in the traveling direction of the asphalt finisher 100. Specifically, the right surrounding image RG is generated by cutting out a part of the input image of the right camera 51R, performing distortion correction, and further performing image rotation processing, and is placed at the right end of the first output image. Ru. Further, the right surrounding image RG includes an image of the right end of the screed 3 and an image of the right end of the hopper 2.

Distortion correction is image processing for correcting apparent distortion and tilt caused by using a wide-angle lens or the like. The image rotation process is image processing for matching the directions of the left surrounding image LG and the right surrounding image RG with the front of the asphalt finisher 100 in the traveling direction (above the screen of the display device 52). In this embodiment, the correspondence between the coordinates on the input image plane and the coordinates on the output image plane regarding the input images of the left camera 51L and the right camera 51R is in a state in which the effects of distortion correction and image rotation processing have been incorporated in advance. and is stored in the input image/output image correspondence map 54a. Note that the distortion correction and image rotation processing may be performed on the hopper image HG.

The illustration image 1CG is a computer graphics of the tractor 1, and is displayed so that the driver can recognize the position of the tractor 1. In this embodiment, the illustration image 1CG is arranged at the lower center of the first output image.

In this way, the display device 52 can display the first output image that shows the asphalt finisher 100 and its vicinity as seen when looking down from above.

In the above embodiment, the hopper image HG, the left surrounding image LG, and the right surrounding image RG are arranged adjacent to each other as separate and independent images. However, the three images may be combined into one continuous image. In this case, image processing may be performed to prevent the image of the object from disappearing in a range where the imaging range of the front camera 51F and the imaging range of the left camera 51L or right camera 51R overlap.

Furthermore, in the above embodiment, each of the hopper image HG, left surrounding image LG, and right surrounding image RG is generated based on an input image captured by one corresponding camera. However, each of the hopper image HG, left surrounding image LG, and right surrounding image RG may be generated based on input images captured by two or more cameras. For example, the left surrounding image LG may be generated based on input images captured by each of the left camera 51L and the left auxiliary camera 51U. Further, the right surrounding image RG may be generated based on input images captured by each of the right camera 51R and the right auxiliary camera 51V.

Next, with reference to FIGS. 4A to 4C, a description will be given of the second output image generated using the respective input images of the left camera 51L, right camera 51R, front camera 51F, left auxiliary camera 51U, and right auxiliary camera 51V. do. 4A to 4C are display examples of the second output image displayed on the display device 52.

The second output image mainly includes a hopper image HG, a left surrounding image LG, a right surrounding image RG, an illustration image 1CG, and a local image SG. In this embodiment, the local image SG is displayed superimposed on the illustration image 1CG. 4A shows an area on the right side of the tractor 1 surrounded by a retaining plate 70 (see FIG. 1B), a side plate 71 (see FIG. 1B), and a mold board 72 (see FIG. 1B). A second output image is shown including a right local image SGR in which a local region is shown. FIG. 4B shows an area on the left side of the tractor 1 surrounded by a retaining plate 70 (see FIG. 1B), a side plate 71 (see FIG. 1B), and a mold board 72 (see FIG. 1B). A second output image is shown including a left local image SGL in which a local area is shown. FIG. 4C shows a second output image including a right local image SGR and a left local image SGL.

The right local image SGR is generated based on the input image of the right auxiliary camera 51V. In this embodiment, the right local image SGR is an image that shows the right local area as seen when looking down on the right local area from the tractor 1. Specifically, the right local image SGR is generated by cutting out a part of the input image of the right auxiliary camera 51V, subjecting it to distortion correction, and further performing image rotation processing. It is arranged along the right edge of 1CG.

The left local image SGL is generated based on the input image of the left auxiliary camera 51U. In this embodiment, the left local image SGL is an image that shows the left local area as seen when looking down at the left local area from the tractor 1. Specifically, the left local image SGL is generated by cutting out a part of the input image of the left auxiliary camera 51U, subjecting it to distortion correction, and further performing image rotation processing. It is arranged along the left edge of 1CG.

The display changeover switch is a first switch for displaying a second output image (see FIG. 4A) that includes the right local image SGR, and a second output image (see FIG. 4B) that includes the left local image SGL. It may be configured to include a second switch for displaying the information. Alternatively, it may be configured only with a switch for displaying the second output image (see FIG. 4C) including the right local image SGR and left local image SGL. Alternatively, it may be configured to include those three switches.

The screw dial may be configured to include a right dial for adjusting the rotational speed of the right screw and a left dial for adjusting the rotational speed of the left screw, so that the rotational speed of the left and right screws can be adjusted simultaneously. It may be configured with only a common dial for adjustment. Alternatively, it may be configured to include those three dials. For example, the highlighting section 50B displays a second output image (see FIG. 4A) including the right local image SGR when the right dial is operated, and displays the left local image SGL when the left dial is operated. A second output image (see FIG. 4B) may be displayed. Alternatively, a second output image (see FIG. 4C) including the right local image SGR and left local image SGL may be displayed when the common dial is operated.

The display frame surrounding the local image SG may be displayed differently from the display frames surrounding each of the hopper image HG, left surrounding image LG, right surrounding image RG, and illustration image 1CG. For example, it may be displayed with different colors, line types, thicknesses, etc., or it may be made to blink.

Next, with reference to FIG. 5, a process in which the image generation system SYS generates an output image (hereinafter referred to as "output image generation process") will be described. FIG. 5 is a flowchart of output image generation processing. The output image includes a first output image and a second output image. The image generation system SYS repeatedly executes this output image generation process at a predetermined control cycle, and selectively generates one of the first output image and the second output image. However, the image generation system SYS may generate both the first output image and the second output image.

First, the output image generation unit 50A of the controller 50 associates the coordinate values on the output image plane with the coordinate values on the input image plane (step S1). In this embodiment, the input image/output image correspondence map 54a is referred to, and the values (for example, brightness values, hue values, saturation values, etc.) of the coordinates on the input image plane that correspond to each coordinate on the output image plane are ) is acquired, and the acquired value is set as the value of each coordinate on the corresponding output image plane.

After that, the controller 50 determines whether all the coordinate values on the output image plane are associated with the coordinate values on the input image plane (step S2).

If it is determined that all the coordinate values have not been associated yet (NO in step S2), the output image generation unit 50A repeats the processes in step S1 and step S2.

If it is determined that all coordinate values have been associated (YES in step S2), the output image generation unit 50A ends the current output image generation process.

Next, referring to FIG. 6, the image generation system SYS performs a process of switching the output image displayed on the display device 52 between the first output image and the second output image (hereinafter referred to as "output image switching process"). .) will be explained. FIG. 6 is a flowchart of output image switching processing. The image generation system SYS repeatedly executes this output image switching process at a predetermined control cycle.

First, the highlighting unit 50B of the controller 50 determines whether or not highlighting has been turned on (step S11). The highlighting unit 50B determines that the highlighting has been turned on, for example, when the display changeover switch serving as the highlighting switch is pressed while the first output image is being displayed on the display device 52. It may be determined that the highlighted display has been turned on when the screw dial is operated.

If it is determined that the highlighting has been turned on (YES in step S11), the highlighting section 50B highlights the local image (step S12). For example, the highlighting section 50B switches the first output image (see FIG. 3) displayed on the display device 52 to the second output image (see FIGS. 4A to 4C) and displays the left local image SGL and the right local image SGL. At least one of the local images SGR is displayed together with surrounding images.

If it is determined that the highlighting has not been turned on (NO in step S11), the highlighting unit 50B switches the first output image displayed on the display device 52 to the second output image. Continue displaying the image.

After that, the highlighting section 50B determines whether or not the highlighting has been turned off (step S13). The highlighting unit 50B determines that the highlighting has been turned off, for example, when the display changeover switch serving as the highlighting switch is pressed while the second output image is being displayed on the display device 52. It may be determined that the highlighted display has been turned off when a predetermined period of time has elapsed since the completion of the operation of the screw dial. Alternatively, it may be determined that the highlighted display has been turned off when a predetermined period of time has elapsed since the display changeover switch was pressed.

If it is determined that the highlighting has been turned off (YES in step S13), the highlighting section 50B stops highlighting the local image (step S14). For example, the highlighting unit 50B switches the second output image displayed on the display device 52 to the first output image and stops highlighting the local image.

If it is determined that the highlighting has not been turned off (NO in step S13), the highlighting unit 50B switches the second output image displayed on the display device 52 to the first output image. Continue displaying the image.

With this configuration, the image generation system SYS can display a local image on the display device 52 in response to a driver's request. By viewing this local image, the driver can view the local area while sitting in the driver's seat 1S without moving around on the tractor 1 or twisting his/her body in the driver's seat 1S to look into the local area. You can check the amount of paving material carried. In addition, the surrounding situation and the amount of paving material held in a local area can be confirmed almost simultaneously without having to move the line of sight significantly. In this way, the asphalt finisher 100 equipped with the image generation system SYS can reduce the driver's fatigue due to the work of checking the amount of cargo carried. As a result, safety regarding the asphalt finisher 100 can be improved.

Next, with reference to FIG. 7, another display example of the second output image will be described. FIG. 7 is another display example of the second output image displayed on the display device 52. The second output image in FIG. 7 differs from the second output image in FIG. 4A in that the right local image SGR is displayed superimposed on the right surrounding image RG instead of the illustration image 1CG, but is the same in other respects. do. Therefore, the explanation of the common parts will be omitted, and the different parts will be explained in detail. The following description relates to the right local image SGR, but applies equally to the left local image SGL.

In the example of FIG. 7, the right local image SGR is generated based on the input image of the right camera 51R, similarly to the right surrounding image RG. Therefore, the right auxiliary camera 51V may be omitted. However, the right local image SGR may be an image generated based on the input image of the right auxiliary camera 51V.

The right local image SGR corresponds to a part of the right surrounding image RG. For example, when the right surrounding image RG is composed of the first image portion RG1 to the eleventh image portion RG11, the right local image SGR in FIG. 7 corresponds to the ninth image portion RG9. Specifically, the image generation system SYS creates a right local image SGR, which is an image obtained by vertically enlarging the ninth image portion RG9, where the seventh image portion RG7 to the eleventh image portion RG11 were displayed. It is displayed in an overlapping manner. That is, the first image portion RG1 to the sixth image portion RG6 are displayed as they are, and the seventh image portion RG7 to the eleventh image portion RG11 are blocked by the right local image SGR and become invisible.

In this way, the image generation system SYS displays the right local image SGR in a superimposed manner on the image portion of the right surrounding image RG in which the right local area was shown. Therefore, the driver can intuitively recognize that the right local area is reflected in the right local image SGR. Furthermore, by displaying the right local image SGR, the right local area can be displayed in a larger size than when the right surrounding image RG is being displayed. Therefore, the state of the right local area to which the paving material is supplied by the right screw can be presented more clearly to the driver.

Next, with reference to FIG. 8, still another display example of the second output image will be described. FIG. 8 is yet another display example of the second output image displayed on the display device 52. The second output image in FIG. 8 differs from the second output image in FIG. 7 in that the right local image SGR is displayed to cover the entire right peripheral image RG instead of a part of it, but in other respects. Common. Therefore, the explanation of the common parts will be omitted, and the different parts will be explained in detail. The following description relates to the right local image SGR, but applies equally to the left local image SGL.

The right local image SGR corresponds to a part of the right surrounding image RG, as in the case of FIG. For example, when the right surrounding image RG is composed of the first image portion RG1 to the eleventh image portion RG11, the right local image SGR in FIG. 8 corresponds to the ninth image portion RG9. Specifically, the image generation system SYS creates a right local image SGR, which is an image obtained by vertically enlarging the ninth image portion RG9, where the first image portion RG1 to the eleventh image portion RG11 were displayed. The images are displayed in an overlapping manner. That is, the first image portion RG1 to the eleventh image portion RG11 are blocked by the right local image SGR and become invisible.

In this way, the image generation system SYS displays the right local image SGR in a superimposed manner over the entire area of the right surrounding image RG, including the image portion where the right local area was shown. Therefore, the driver can intuitively recognize that the right local area is reflected in the right local image SGR. Furthermore, by displaying the right local image SGR having a size spanning the entire length of the display device 52 in the vertical direction, the right local area can be displayed even larger than in the case of the second output image in FIG. 7 . Therefore, the state of the right local area to which the paving material is supplied by the right screw can be presented to the driver more clearly.

Next, with reference to FIG. 9, still another display example of the second output image will be described. FIG. 9 is yet another display example of the second output image displayed on the display device 52. The second output image in FIG. 9 is different from the second output image in FIG. It is different from the output image but has other points in common. Therefore, the explanation of the common parts will be omitted, and the different parts will be explained in detail. The following description relates to the right local image SGR, but applies equally to the left local image SGL.

The right local image SGR corresponds to the entire right surrounding image RG, unlike the case in FIG. For example, when the right surrounding image RG is composed of the first image portion RG1 to the eleventh image portion RG11, the right local image SGR in FIG. It is generated by enlarging or reducing it. Specifically, each of the first image portion RG1 to the sixth image portion RG6 and the eleventh image portion RG11 is reduced in the vertical direction, and each of the seventh image portion RG7 to the tenth image portion RG10 is reduced in the vertical direction. It consists of enlarged images. That is, unlike the cases of FIGS. 7 and 8, the scene shown in the right surrounding image RG can be continuously viewed even when the right local image SGR is displayed.

The indicator BG is a graphical image representing the enlargement/reduction state of the image portion constituting the local image SG with respect to the corresponding image portion constituting the surrounding image. In the example of FIG. 9, the right indicator BGR represents the enlargement/reduction state of the image portion forming the right local image SGR with respect to the corresponding image portion forming the right surrounding image RG. Specifically, the right indicator BGR is a bar-shaped indicator that extends in the vertical direction and is composed of 11 rectangular segments corresponding to each of the first image portion RG1 to the eleventh image portion RG11, and is displayed at the right end of the screen. ing. If the left indicator is displayed, the left indicator may be displayed on the left edge of the screen. The left indicator represents the scaling state of the image portion constituting the left local image SGL with respect to the corresponding image portion constituting the left surrounding image LG. In either case, the bar-shaped indicator indicates that the longer the rectangular segment is vertically, the greater the magnification rate is, and the shorter the rectangular segment is vertically, the greater the reduction rate is. However, the display of the indicator BG may be omitted.

In this way, the image generation system SYS displays the right local image SGR in a superimposed manner over the entire area of the right surrounding image RG, including the image portion where the right local area was shown. Therefore, the driver can intuitively recognize that the right local area is reflected in the right local image SGR. Furthermore, as in the case of FIG. 8, by displaying the right local image SGR over the entire length of the display device 52 in the vertical direction, the right local area can be displayed larger than in the case of the second output image in FIG. Therefore, the state of the right local area to which the paving material is supplied by the right screw can be presented more clearly to the driver. Furthermore, unlike the case of FIG. 8, the driver can continuously view the scene reflected in the right surrounding image RG while displaying the right local area in a large size. Therefore, the driver can check the state of the right local area while monitoring the worker on the right side of the asphalt finisher 100, for example.

With the above configuration, the image generation system SYS allows the driver to intuitively recognize the positional relationship between the asphalt finisher 100 and workers working around it based on input images acquired by multiple cameras. can generate output images.

The image generation system SYS also generates a hopper image HG, a left surrounding image LG, and a right surrounding image RG so that the driver can be presented with an image that shows the asphalt finisher 100 and its vicinity when viewed from above. , and an illustration image 1CG are displayed. Thereby, the driver can visually recognize the blind spot around the asphalt finisher 100 without leaving the driver's seat 1S. As a result, the image generation system SYS can improve the safety and operability of the asphalt finisher 100. Specifically, the image generation system SYS can present to the driver the remaining amount of paving material in the hopper 2, the position of a feature (for example, a manhole) on the road surface to be paved, and the like. Furthermore, the image generation system SYS can present the positions of workers and the like working around the asphalt finisher 100 to the driver. Therefore, the driver can perform various operations such as opening and closing the hopper 2, expanding and contracting the screed 3, and raising and lowering the screed 3 after confirming the positions of the workers and the like by looking at the display device 52. Furthermore, if the driver senses danger in the positional relationship between the worker and the hopper, screed, or dump truck, the driver can cancel various operations, stop the asphalt finisher, etc.

The image generation system SYS also displays a surrounding image that shows the surroundings of the asphalt finisher 100, and emphasizes a local image that shows the area where paving material is supplied by the screw SC. indicate. Emphasizing the local image includes displaying the local image in a separate display frame, enlarging the local image, changing the display mode of the local image display frame, and the like. Through this processing, the image generation system SYS can present the situation in a predetermined local area to the driver in an easy-to-understand manner, in addition to the situation around the asphalt finisher 100.

The image generation system SYS includes a right camera 51R as a first camera that images an area on the right side of the asphalt finisher 100, and a right auxiliary camera 51V as a second camera that images an area where paving material is supplied by a right screw. It may also have the following. In this case, the right surrounding image RG constituting the surrounding image is generated based on the image captured by the right camera 51R, and the right local image SGR constituting the local image is generated based on the image captured by the right auxiliary camera 51V. may also be generated.

The image generation system SYS also includes a left camera 51L as a first camera that images an area on the left side of the asphalt finisher 100, and a left auxiliary camera 51L as a second camera that images an area where paving material is supplied by a left screw. It may also include a camera 51U. In this case, the left surrounding image LG that constitutes the surrounding image is generated based on the image captured by the left camera 51L, and the left local image SGL that constitutes the local image is generated based on the image captured by the left auxiliary camera 51U. may also be generated.

The image generation system SYS may include a right camera 51R that images the area on the right side of the asphalt finisher 100 and the right local area where the paving material is supplied by the right screw. In this case, the right surrounding image RG and the right local image SGR may be generated based on the image captured by the right camera 51R. In this case, the right auxiliary camera 51V may be omitted.

The image generation system SYS may also include a left camera 51L that images the area to the left of the asphalt finisher 100 and the left local area to which the paving material is supplied by the left screw. In this case, the left surrounding image LG and the left local image SGL may be generated based on the image captured by the left camera 51L. In this case, the left auxiliary camera 51U may be omitted.

The local image may be displayed so as not to overlap with the surrounding image, as shown in FIGS. 4A to 4C, for example, or may be displayed so as to overlap with the surrounding image, as shown in FIGS. 6 to 8, for example. .

The display device 52 may display an indicator BG representing the enlargement/reduction state of the local image SG. By looking at the indicator BG, the driver can instantly grasp the enlargement/reduction state of each image portion constituting the local image SG.

BACKGROUND ART Conventionally, an asphalt finisher is known in which images captured by a front camera, a left camera, and a right camera are arranged and displayed around computer graphics of a tractor (see Patent Document 1). The front camera is attached to the upper front end of the tractor to take an image of the inside of the hopper in front of the tractor. The left camera is attached to the upper end of the left side of the tractor to image the space on the left side of the asphalt finisher. The right camera is attached to the upper end of the right side of the tractor to image the space on the right side of the asphalt finisher.

However, in the above configuration, the space around the front end of the hopper wing becomes a blind spot of the hopper wing when viewed from the front camera, left camera, and right camera. Therefore, the operator of the asphalt finisher cannot grasp the state of the space around the front end of the hopper wing even by looking at the displayed image. For spaces that cannot be visually recognized in images, the operator needs to visually confirm safety directly.

In view of the above, it is desired to provide a road machine that further reduces the range that cannot be visually recognized in images.

FIG. 10 is a side view of an asphalt finisher 100, which is an example of a road machine according to an embodiment of the present invention. FIG. 11 is a top view of the asphalt finisher 100. The asphalt finisher 100 mainly includes a tractor 1, a hopper 2, and a screed 3. In the following, the direction of the hopper 2 as seen from the tractor 1 (+X direction) is referred to as the front, and the direction of the screed 3 as seen from the tractor 1 (−X direction) as the rear.

The tractor 1 is a vehicle for driving the asphalt finisher 100. In this embodiment, the tractor 1 rotates the rear wheels 5 using the rear wheel running hydraulic motor, and rotates the front wheels 6 using the front wheel running hydraulic motor to move the asphalt finisher 100. The hydraulic motor for running the rear wheels and the hydraulic motor for running the front wheels are supplied with hydraulic oil from the hydraulic pump and rotate. The rear wheels 5 and front wheels 6 may be replaced by crawlers. Moreover, the tractor 1 includes a canopy 1C. The canopy 1C is attached to the top of the tractor 1.

Controller 50 is a control device that controls asphalt finisher 100. In this embodiment, the controller 50 is comprised of a microcomputer including a CPU, a volatile storage device, a nonvolatile storage device, etc., and is mounted on the tractor 1. Various functions of the controller 50 are realized by the CPU executing programs stored in a nonvolatile storage device.

The hopper 2 is a mechanism for receiving paving material, and mainly includes a hopper wing 20 and a hopper cylinder 24. In this embodiment, the hopper 2 is installed on the front side of the tractor 1 and receives the paving material in the hopper wing 20. The hopper wing 20 includes a left hopper wing 20L that can be opened and closed in the Y-axis direction (vehicle width direction) by a left hopper cylinder 24L, and a right hopper wing 20R that can be opened and closed in the Y-axis direction (vehicle width direction) by a right hopper cylinder 24R. including. The asphalt finisher 100 normally receives paving material (for example, an asphalt mixture) from the bed of a dump truck with the hopper wing 20 fully open. FIG. 11 shows the hopper wing 20 in a fully open state. When the amount of paving material in the hopper 2 decreases, the hopper wing 20 is closed, and the paving material that was near the inner wall of the hopper 2 is collected in the center of the hopper 2. This is to enable the conveyor CV located in the center of the hopper 2 to continuously feed paving material to the rear side of the tractor 1. That is, this is to maintain the state in which the paving material is accumulated on the conveyor CV. Thereafter, the paving material fed to the rear side of the tractor 1 is spread in the vehicle width direction on the rear side of the tractor 1 and the front side of the screed 3 by the screw SC. In this embodiment, the screw SC is in a state in which extension screws are connected on the left and right sides.

The screed 3 is a mechanism for leveling the paving material. In this embodiment, the screed 3 includes a front screed 30 and a rear screed 31. The screed 3 is a floating screed that is pulled by the tractor 1, and is connected to the tractor 1 via a leveling arm 3A. The rear screed 31 includes a left rear screed 31L and a right rear screed 31R. The left rear screed 31L is expanded and contracted in the vehicle width direction using the left screed telescopic cylinder 26L, and the right rear screed 31R is expanded and contracted in the vehicle width direction using the right screed telescopic cylinder 26R.

The imaging device 51 is a device that acquires images. In this embodiment, the imaging device 51 is a monocular camera, and is connected to the controller 50 wirelessly or by wire. The controller 50 can generate a bird's-eye view image, for example, by performing viewpoint conversion processing on the image captured by the imaging device 51. The bird's-eye view image is, for example, an image of the space around the asphalt finisher 100 virtually viewed from approximately directly above. The imaging device 51 may be a stereo camera. In this embodiment, the imaging device 51 includes a front camera 51F, a left camera 51L, a right camera 51R, and a rear camera 51B. The rear camera 51B may be omitted.

The front camera 51F images the space in front of the asphalt finisher 100. In this embodiment, the front part of the tractor 1 is configured so that an image of the inside of the hopper 2, which is a blind spot, can be captured from the viewpoint of the driver seated in the driver's seat 1S (hereinafter referred to as "driver's seat viewpoint"). attached to the bonnet. It may be attached to the front edge of the top plate of the canopy 1C. A gray area Z1 in FIG. 11 indicates the imaging range of the front camera 51F.

The left camera 51L images the space to the left of the asphalt finisher 100. In this embodiment, a rod member extends in the +Y direction (leftward) from the left edge of the top plate of the canopy 1C so that an image of the space outside the left hopper wing 20L in the vehicle width direction, which is a blind spot from the viewpoint of the driver's seat, can be captured. It is attached to the tip of BL. It may be attached to the tip of a bar member BL extending from the right side of the tractor 1 in the +Y direction (leftward). The left camera 51L is attached, for example, so as to protrude outward (to the left) in the vehicle width direction from the left end of the left hopper wing 20L in a fully open state. A gray area Z2 in FIG. 11 indicates the imaging range of the left camera 51L.

The right camera 51R images the space to the right of the asphalt finisher 100. In this embodiment, in order to capture an image of the space outside the right hopper wing 20R in the vehicle width direction, which is a blind spot from the perspective of the driver's seat, a rod is provided that extends in the -Y direction (to the right) from the right edge of the top plate of the canopy 1C. It is attached to the tip of member BR. It may be attached to the tip of a rod member BR extending from the right side of the tractor 1 in the −Y direction (rightward). The right camera 51R is attached, for example, so as to protrude outward (to the right) in the vehicle width direction from the right end of the right hopper wing 20R in a fully open state. A gray area Z3 in FIG. 11 indicates the imaging range of the right camera 51R.

The rod members BL, BR are preferably configured to be removable. It may be configured to be expandable and contractible. This is to cope with the case where the asphalt finisher 100 is transported by a trailer or the like.

The rear camera 51B images the space behind the asphalt finisher 100. In this embodiment, it is attached to the rear edge of the top plate of the canopy 1C so that the space behind the screed 3, which is a blind spot from the viewpoint of the driver's seat, can be imaged. A gray area Z4 in FIG. 11 indicates the imaging range of the rear camera 51B.

The imaging range of the front camera 51F and the imaging range of the left camera 51L may overlap. Further, the imaging range of the rear camera 51B and the imaging range of the left camera 51L do not need to overlap. The same applies to the imaging range of the right camera 51R.

The controller 50 generates a bird's-eye view image by converting and combining the images taken by the front camera 51F, left camera 51L, right camera 51R, and rear camera 51B. The bird's-eye view image is a virtual view of the space inside the hopper 2, the space to the left of the left hopper wing 20L, the space to the right of the right hopper wing 20R, and the space behind the screed 3 from almost directly above. and a computer graphics image of the asphalt finisher 100 (hereinafter referred to as a "model image"). The controller 50 may generate a bird's-eye view image by converting and combining the images taken by each of the three cameras, the front camera 51F, the left camera 51L, and the right camera 51R. That is, the bird's-eye view image may be generated without using the rear camera 51B.

The display device 52 is a device that displays various images. In this embodiment, the display device 52 is a liquid crystal display, and is connected to the controller 50 wirelessly or by wire. The display device 52 can display images captured by each of the plurality of imaging devices 51, and is arranged at a position where the driver sitting in the driver's seat 1S can easily view the image. It may be located at the rear controller. The controller 50 displays, on the display device 52, an image generated by performing viewpoint conversion processing on the image captured by the imaging device 51, for example.

Next, with reference to FIG. 12, the display system GS installed in the asphalt finisher 100 will be described. FIG. 12 is a block diagram showing an example of the configuration of the display system GS. The display system GS mainly includes a controller 50, an imaging device 51, a display device 52, an information acquisition device 53, a storage device 54, and the like. The display system GS, for example, generates an image for display (hereinafter referred to as an "output image") based on an image captured by the imaging device 51 (hereinafter referred to as an "input image"), and outputs the output image. It is displayed on the display device 52.

The information acquisition device 53 acquires information and outputs the acquired information to the controller 50. The information acquisition device 53 includes, for example, at least one of a hopper cylinder stroke sensor, a screed telescopic cylinder stroke sensor, a steering angle sensor, a travel speed sensor, a positioning sensor, and the like. The hopper cylinder stroke sensor detects the stroke amount of the hopper cylinder 24. The screed telescopic cylinder stroke sensor detects the stroke amount of the screed telescopic cylinder 26. The steering angle sensor detects the steering angle of the front wheels 6. The traveling speed sensor detects the traveling speed of the asphalt finisher 100. The positioning sensor is, for example, a GNSS compass, and detects the position (latitude, longitude, altitude) and orientation of the asphalt finisher 100.

The storage device 54 is a device for storing various information. In this embodiment, the storage device 54 is a nonvolatile storage device that stores an input image/output image correspondence map 54a in a referable manner.

The input image/output image correspondence map 54a stores the correspondence between coordinates on the input image plane and coordinates on the output image plane. The correspondence relationship is set in advance based on various parameters such as the optical center of the imaging device 51, focal length, CCD size, optical axis direction vector, camera horizontal direction vector, and projection method so as to realize the desired viewpoint conversion. There is. The correspondence relationship is set so that apparent distortion or tilt does not appear in the output image.

The controller 50 includes a viewpoint conversion section 50a and an auxiliary line creation section 50b. The viewpoint conversion unit 50a and the auxiliary line creation unit 50b are configured by software, hardware, or firmware.

The viewpoint conversion unit 50a is a functional element that generates an output image. In this embodiment, the input image/output image correspondence map 54a stored in the storage device 54 is referred to, and the coordinates on the input image plane where the input image imaged by the imaging device 51 is located and the overhead image as the output image are determined. The coordinates of the location on the output image plane are associated with each other. Specifically, the viewpoint conversion unit 50a converts the output image by associating the value of each pixel in the input image (for example, brightness value, hue value, saturation value, etc.) with the value of each pixel in the output image. generate.

The auxiliary line creation unit 50b is a functional element that creates auxiliary lines that are superimposed and displayed on the output image. In this embodiment, the auxiliary line creation unit 50b creates auxiliary lines to match the bird's-eye view image generated by the viewpoint conversion unit 50a. The auxiliary lines include, for example, an auxiliary line indicating an expected paving trajectory which is the expected trajectory of the end of the screed 3, an auxiliary line indicating an expected travel trajectory which is the expected trajectory of the wheels, and the like.

In this embodiment, the controller 50 uses the viewpoint conversion unit 50a to refer to the input image/output image correspondence map 54a. Then, the values (e.g., brightness value, hue value, saturation value, etc.) of the coordinates on the input image plane corresponding to each coordinate on the output image plane are obtained, and the obtained values are applied to the corresponding coordinates on the input image plane. Adopted as the value of each coordinate on the output image plane.

Thereafter, the controller 50 determines whether all coordinate values on the output image plane have been associated with coordinate values on the input image plane. If it is determined that not all coordinate values have been associated yet, the above-described process is repeated.

On the other hand, if it is determined that all the coordinate values have been associated, the controller 50 causes the auxiliary line creation unit 50b to superimpose and display an auxiliary line indicating the expected pavement trajectory, an auxiliary line indicating the expected travel trajectory, etc. on the output image. do. In this embodiment, the position on the output image at which the auxiliary line is superimposed is set in advance, but it may be dynamically derived. Further, after displaying the auxiliary line, the controller 50 may associate the coordinates on the input image plane with the coordinates on the output image plane.

Next, with reference to FIGS. 13A and 13B, input images captured by each of the four imaging devices 51 (front camera 51F, left camera 51L, right camera 51R, and rear camera 51B) mounted on the asphalt finisher 100 The bird's-eye view image generated using . FIGS. 13A and 13B are diagrams showing examples of overhead images. Specifically, FIG. 13A shows an example of an overhead image displayed on the display device 52. FIG. 13B shows the segmentation of the input image used to generate the overhead image of FIG. 13A.

The bird's-eye view image in FIG. 13A is an image of the space around the asphalt finisher 100 virtually viewed from approximately directly above. The bird's-eye view image in FIG. 13A mainly includes an image G1 (see the shaded area) generated by the viewpoint conversion unit 50a and a model image CG1.

The model image CG1 is an image representing the asphalt finisher 100, and includes a model image CGa of the hopper 2, a model image CGb of the tractor 1, and a model image CGc of the screed 3.

The model image CGa of the hopper 2 includes a model image WL of the left hopper wing 20L and a model image WR of the right hopper wing 20R. The shape of the model images WL and WR changes depending on the output of the hopper cylinder stroke sensor. FIG. 13A shows a model image CGa when each of the left hopper wing 20L and the right hopper wing 20R is fully open. A part of the bird's-eye view image (an image of the inside of the hopper 2 virtually viewed from almost directly above) is arranged in the diagonally shaded area between the model image WL and the model image WR. The model image CGa of the hopper 2 may be omitted. In this case, a part of the overhead image based on the image captured by the front camera 51F is arranged.

The model image CGc of the screed 3 includes a model image SL of the left rear screed 31L and a model image SR of the right rear screed 31R. The shape of the model images SL and SR changes depending on the output of the screed telescopic cylinder stroke sensor. FIG. 13A shows a model image CGc when each of the left rear screed 31L and the right rear screed 31R is fully expanded. The model image CGc of the screed 3 may be omitted. In this case, a part of the overhead image based on the image captured by the rear camera 51B is arranged.

The image G1 is an image generated using input images captured by each of the four imaging devices 51. In the present embodiment, the image G1 includes an image Ga of a worker located at the left front of the asphalt finisher 100 and an image Gb of a manhole cover located at the right front of the asphalt finisher 100. As shown in FIG. 13B, the controller 50 generates the image G1 by combining the front image R1, the left image R2, the right image R3, and the rear image R4. The image Ga of the worker is included in the left image R2, and the image Gb of the manhole cover is included in the right image R3.

The front image R1 is an image generated based on the input image captured by the front camera 51F. In this embodiment, the previous image R1 includes an image showing the inside of the hopper 2 when looking down from the tractor 1 side. The controller 50 generates the front image R1 by cutting out a part of the input image captured by the front camera 51F and performing viewpoint conversion processing. The previous image R1 is arranged between the model image WL and the model image WR and above them. The shape of the previous image R1 may change according to changes in the shapes of the model images WL and WR.

The left image R2 is an image generated based on the input image captured by the left camera 51L. In this embodiment, the left image R2 includes an image of the space outside (on the left side) in the vehicle width direction of the left hopper wing 20L. The controller 50 generates the left image R2 by cutting out a part of the input image captured by the left camera 51L and subjecting it to viewpoint conversion processing by the viewpoint conversion unit 50a. Left image R2 is placed on the left side of model image CG1. The shape of the left image R2 may change according to changes in the shapes of the model images WL and SL.

The right image R3 is an image generated based on the input image captured by the right camera 51R. In this embodiment, the right image R3 includes an image of the space outside (right side) in the vehicle width direction of the right hopper wing 20R. The controller 50 generates the right image R3 by cutting out a part of the input image captured by the right camera 51R and subjecting it to viewpoint conversion processing by the viewpoint conversion unit 50a. Right image R3 is placed on the right side of model image CG1. The shape of right image R3 may change according to changes in the shapes of model images WR and SR.

The rear image R4 is an image generated based on the input image captured by the rear camera 51B. In this embodiment, the rear image R4 includes an image showing the screed 3 looking down from the tractor 1 side. The controller 50 generates the rear image R4 by cutting out a part of the input image captured by the rear camera 51B and performing viewpoint conversion processing. The rear image R4 is arranged below the model image CGc of the screed 3. The shape of the rear image R4 may change according to changes in the shapes of the model images SL and SR.

In this embodiment, the correspondence between the coordinates on the input image plane and the coordinates on the output image plane of each of the four cameras is determined by the input image/output image correspondence map 54a with the effect of viewpoint conversion processing taken in advance. is stored in Therefore, the controller 50 can associate coordinates on a plurality of input image planes with coordinates on an output image plane simply by referring to the input image/output image correspondence map 54a. As a result, controller 50 can generate and display output images with a relatively low computational load.

Further, in this embodiment, each of the front image R1, left image R2, right image R3, and rear image R4 is generated based on an input image captured by one corresponding camera. However, the present invention is not limited to this configuration. For example, each of the front image R1, left image R2, right image R3, and rear image R4 may be generated based on input images captured by two or more cameras.

As described above, the asphalt finisher 100 includes a plurality of imaging devices 51 attached to the tractor 1, a controller 50 that converts viewpoints of images captured by each of the plurality of imaging devices 51, and synthesizes them to generate an overhead image. A display device 52 that displays an overhead image on one screen is provided. The bird's-eye view image is configured to include an image of the space around the asphalt finisher 100 virtually viewed from substantially directly above. Therefore, it is possible to further reduce blind spots, which are areas that cannot be visually recognized in the output image. As a result, for example, the state of the space around the front end of the hopper wing 20 can be presented to the operator of the asphalt finisher 100. Alternatively, the state of the paving material inside the hopper 2, the state of the space behind the screed 3, etc. can be presented to the operator.

Furthermore, by presenting an overhead image, the asphalt finisher 100 can improve visibility, safety, operability, and workability. Specifically, the asphalt finisher 100 allows the operator to intuitively recognize the remaining amount of paving material in the hopper 2, the position of features (for example, manholes) on the road surface to be paved, etc. . Further, the operator can intuitively recognize the position of the worker working around the hopper 2. Therefore, the operator can perform various operations such as opening and closing the hopper wing 20 after confirming the positions of features and workers by looking at the bird's-eye view image.

Next, another example of the output image will be described with reference to FIGS. 14A and 14B. 14A and 14B show an example in which auxiliary lines are displayed superimposed on the image G1 generated by the viewpoint conversion unit 50a. Specifically, FIG. 14A shows an overhead image of the asphalt finisher 100 that is about to move straight. FIG. 14B shows an overhead image of the asphalt finisher 100 about to turn to the right.

In the example of FIGS. 14A and 14B, the auxiliary line creation unit 50b creates auxiliary lines L1 and L2 based on the respective outputs of the steering angle sensor and the traveling speed sensor. The auxiliary line L1 is the expected travel trajectory of the left rear wheel 5L, and the auxiliary line L2 is the expected travel trajectory of the right rear wheel 5R. In this embodiment, the auxiliary lines L1 and L2 are derived based on the current steering angle and travel speed. It may also be derived based only on the steering angle. The auxiliary lines L1 and L2 represent, for example, a travel locus for a period from the current moment until a predetermined period of time (for example, several tens of seconds) has elapsed.

Further, the auxiliary line creation unit 50b creates auxiliary lines L3 and L4 by additionally referring to the output of the screed telescopic cylinder stroke sensor. The auxiliary line L3 is the expected trajectory of the left end of the left rear screed 31L, and the auxiliary line L4 is the expected trajectory of the right end of the right rear screed 31R. In this embodiment, the auxiliary lines L3 and L4 are derived based on the current steering angle, travel speed, and stroke amount of the screed telescopic cylinder 26. The auxiliary lines L3 and L4 represent, for example, a trajectory for a period from the current moment until a predetermined time (for example, several tens of seconds) has elapsed.

Further, the auxiliary line creation unit 50b creates auxiliary lines L5 and L6 based on the road design data and the output of the positioning sensor. The road design data is data related to the road to be constructed, and is stored in advance in, for example, a nonvolatile storage device. The road design data includes, for example, data regarding the positions of features on the road surface to be paved. The auxiliary line L5 represents the left edge of the road to be constructed, and the auxiliary line L6 represents the right edge of the road to be constructed.

Further, the controller 50 superimposes and displays images of features such as manholes on the bird's-eye view image based on the road design data and the output of the positioning sensor. In the example of FIGS. 14A and 14B, the controller 50 displays the model image CG2 of the manhole cover in a superimposed manner on the bird's-eye view image.

The operator of the asphalt finisher 100 can check the current steering angle, traveling speed, expansion/contraction amount of the rear screed 31, etc. of the asphalt finisher 100 appropriately by looking at the overhead image on which the auxiliary lines etc. are superimposed as described above. You can judge whether it is there or not. For example, by looking at the bird's-eye view image in FIG. 14A, the operator can recognize that if the asphalt finisher 100 continues to move straight, the right rear wheel 5R will run onto the manhole cover, and the road will not be paved as designed. . On the other hand, by looking at the bird's-eye view image in FIG. 14B, the operator can avoid the right rear wheel 5R running onto the manhole cover by maintaining the current steering condition (steering wheel turned to the right) and traveling speed. , and it can be recognized that the road will be paved as designed.

In the examples shown in FIGS. 14A and 14B, the controller 50 displays the predicted travel trajectory of the rear wheels 5, but may also display the predicted travel trajectory of the front wheels 6. The travel trajectory may also be displayed.

With the above-described configuration, the controller 50 can present in advance to the operator the movement of the asphalt finisher 100 during the period from the current moment until a predetermined time has elapsed, in addition to the effect of the bird's-eye view image described with reference to FIGS. 13A and 13B. This additional effect can be achieved.

The preferred embodiments of the invention have been described above. However, the invention is not limited to the embodiments described above. Various modifications, substitutions, etc. may be applied to the embodiments described above without departing from the scope of the present invention. Furthermore, the features described with reference to the above-described embodiments may be combined as appropriate unless technically contradictory.

For example, in the above-described embodiment, the image portion constituting the local image SG is scaled only in the vertical direction, but it may also be scaled only in the horizontal direction, or it may be scaled in both the vertical and horizontal directions. Good too. If the image portion is scaled only laterally, the indicator BG may be a bar-shaped indicator extending laterally. If the image portion is scaled vertically and horizontally, the indicator BG may be a matrix-like indicator extending vertically and horizontally.

Furthermore, the asphalt finisher 100 may be a goose asphalt finisher using a goose asphalt mixture. The image generation system SYS may be mounted on a goose asphalt finisher that uses goose asphalt mix.

This application claims priority based on Japanese Patent Application No. 2017-153668 filed on August 8, 2017, and priority based on Japanese Patent Application No. 2017-164687 filed on August 29, 2017. The entire contents of these Japanese patent applications are hereby incorporated by reference into this application.

1...Tractor 1S...Driver's seat 2...Hopper 3...Screed 3A...Leveling arm 5...Rear wheel 6...Front wheel 20...Hopper wing 20L...Left hopper Wing 20R...Right hopper wing 24...Hopper cylinder 24L...Left hopper cylinder 24R...Right hopper cylinder 26...Screed telescopic cylinder 26L...Left screed telescopic cylinder 26R...Right screed telescopic Cylinder 30...Front screed 31...Rear screed 31L...Left rear screed 31R...Right rear screed 50...Controller 50A...Output image generation section 50B...Highlight display section 50a...Viewpoint conversion section 50b...Auxiliary line creation section 51...Imaging device 51B...Rear camera 51F...Front camera 51L...Left camera 51R...Right camera 51U...Left Auxiliary camera 51V... Right auxiliary camera 52... Display device 53... Information acquisition device 54... Storage device 54a... Input image/output image correspondence map 55... Input device 65... Operation Panel 70... Retaining plate 71... Side plate 72... Mold board 100... Asphalt finisher BL, BR... Bar member CV... Conveyor SC... Screw SYS... Image generation system

Claims (5)

tractor and
a hopper installed on the front side of the tractor and receiving paving material in a hopper wing;
a conveyor that feeds the paving material in the hopper to the rear side of the tractor;
a screw that spreads the paving material fed by the conveyor on the rear side of the tractor;
a screed for leveling the paving material spread by the screw on the rear side of the screw;
a plurality of imaging devices attached to the tractor;
a control device;
a display device;
An information acquisition device;
The control device generates an overhead image of the space around the road machine based on the image captured by the imaging device, creates an expected travel trajectory of the wheels based on the information acquired by the information acquisition device, and generates an overhead image of the space around the road machine based on the image captured by the imaging device. displaying an image, the predicted travel trajectory, and an image of a feature on the road on the display device;
road machinery. The control device creates the predicted travel trajectory of the rear wheels based on at least a steering angle of the front wheels.
The road machine according to claim 1. The control device displays an image of the feature in a superimposed manner on the bird's-eye view image based on data regarding the position of the feature.
The road machine according to claim 1. The control device creates an auxiliary line representing an edge of the road to be constructed, and causes the display device to display the auxiliary line.
The road machine according to claim 1. The control device creates a predicted trajectory of the end of the screed, and causes the display device to display the predicted trajectory.
The road machine according to claim 1.

Images