JP6243806B2 - Work vehicle surrounding monitoring device and program for surrounding monitoring device - Google Patents

Description

  The present invention relates to a surrounding monitoring device for a work vehicle and a program for the surrounding monitoring device.

  At the pavement construction site, compaction work is performed using compaction road machines such as road rollers and tire rollers. In this compaction work, since the work is performed in cooperation with workers around the road machine and other work machines, it is necessary for the operator to always monitor the situation around the road machine.

  On the other hand, an apparatus as described in Patent Document 1 is known as a system for monitoring the surroundings of a vehicle. That is, a plurality of cameras that monitor the surroundings of the vehicle are installed, and the images of these cameras are converted into an upper viewpoint image and combined to provide an overhead view image around the vehicle.

WO00 / 07373

  By the way, in the work by the road machine, for example, the above-described paving work, the road machine is moved along the boundary line so that the compaction is surely performed to the boundary part with the road shoulder. Therefore, careful operation is required near the boundary line. At the same time, it is necessary to constantly monitor the situation around the road machine. Therefore, it is necessary to frequently move the viewpoint between the vicinity of the boundary line of the road shoulder and the periphery of the road machine, and this poor workability is a great burden on the operator of the road machine.

  In addition, when the above-described surrounding monitoring system is applied, when performing a bird's-eye view display in which a camera image is converted into an image of a viewpoint (upper viewpoint) from directly above the vehicle and combined into one image, image synthesis is performed. Since the position of the surface is not taken into account, the road shoulder may be difficult to see due to the effect of the image synthesis surface processing, and there is a problem that it is difficult to drive along the road shoulder.

A work vehicle surrounding monitoring apparatus according to a first aspect of the present invention confirms a front confirmation image for confirming the front of a vehicle and a rear of the vehicle based on images of the surroundings of the vehicle taken by a plurality of cameras provided on the vehicle. an image generator for generating a rear view picture image for an image selection instruction input unit image selection command is input for selecting one of the generated the front confirmation image and the rear view image, the image selection instruction An output unit that outputs an image selected by the input unit to a display unit, wherein the front confirmation image includes a front viewpoint image generated based on an image acquired by a camera in front of the vehicle, a camera in front of the vehicle, synthesized from a first upper viewpoint overhead image generated based on each image acquired by the vehicle left side of the camera and the vehicle right side of the camera, and, imitating a work vehicle in the front check image Along a line parallel to the street said vehicle front to the vehicle front of the vehicle icon, the forward viewpoint image and the first upper viewpoint overhead view image are synthesized to be connected, the rearview image, vehicle rear camera From the rear viewpoint image generated based on the image acquired in step 2 and the second upper viewpoint overhead image generated based on the images acquired by the camera behind the vehicle, the camera on the left side of the vehicle, and the camera on the right side of the vehicle. And the rear viewpoint image and the second upper viewpoint overhead image are connected along a line passing through the vehicle rear surface of the vehicle icon in the rear confirmation image and parallel to the vehicle rear surface. The first upper viewpoint overhead image is based on an upper right viewpoint image that is an upper viewpoint image based on an image acquired by the right side camera of the vehicle and an image acquired by the left side camera of the vehicle. The front side of the area occupied by the work vehicle of the display unit is a composite line of the upper left viewpoint image that is the upper viewpoint image and the front upper viewpoint image that is the upper viewpoint image based on the image acquired by the camera in front of the vehicle The upper right viewpoint image, the upper left viewpoint image, and the upper front viewpoint image so that the scales of the upper right viewpoint image, the upper left viewpoint image, and the front upper viewpoint image are the same. The second upper viewpoint overhead image generated by combining the upper upper viewpoint image, which is an upper viewpoint image based on the image acquired by the camera on the right side of the vehicle, and the upper side based on the image acquired by the camera on the left side of the vehicle. A composite line of the upper left viewpoint image that is the viewpoint image and the rear upper viewpoint image that is the upper viewpoint image based on the image acquired by the camera behind the vehicle is the work of the display unit. The upper right viewpoint image, the upper left viewpoint image so as to coincide with the rear line of the area occupied by the vehicle, and so that the scales of the upper right viewpoint image, the upper left viewpoint image, and the rear upper viewpoint image are the same. and it is produced by combining the upper viewpoint image after such, the image selection instruction input section, Ru image switching switch der accept the image selection command from the operator.
The invention of claim 2 is the surroundings monitoring device for a work vehicle according to claim 1, wherein the image generating unit is configured so that the vehicle icon is displayed at a predetermined display area of the display unit, the front check image, or The back confirmation image and the vehicle icon are combined.
The invention according to claim 3 is a front confirmation image for confirming the front of the vehicle and a rear confirmation image for confirming the rear of the vehicle based on images around the vehicle captured by a plurality of cameras provided on the vehicle. An image generation function for generating an image selection function, an image selection function for receiving an image selection command for selecting one of the generated front confirmation image and the rear confirmation image via an image changeover switch for receiving an instruction from an operator, and the image selection function. A program for monitoring the surroundings of a work vehicle for causing a computer to output an image whose selection has been received to a display unit , wherein the forward confirmation image is based on an image acquired by a camera in front of the vehicle Generated based on the front viewpoint image generated by the camera, the camera in front of the vehicle, the camera on the left side of the vehicle, and the camera on the right side of the vehicle. The front viewpoint image along a line parallel to the vehicle front surface of the vehicle icon of the vehicle icon imitating the work vehicle in the front confirmation image. And the first upper viewpoint overhead image are combined so that the rear confirmation image is generated based on the image acquired by the camera behind the vehicle, the rear viewpoint image, the vehicle rear camera, and the vehicle left side. And the second upper viewpoint overhead image generated on the basis of the images acquired by the camera on the right side and the camera on the right side of the vehicle, and pass through the vehicle rear surface of the vehicle icon in the rear confirmation image on the vehicle rear surface. The rear viewpoint image and the second upper viewpoint overhead image are combined so as to be connected along a parallel line, and the first upper viewpoint overhead image is obtained by a camera on the right side of the vehicle. Upper right viewpoint image that is an upper viewpoint image based on the obtained image, upper left viewpoint image that is an upper viewpoint image based on an image acquired by the camera on the left side of the vehicle, and upper direction based on an image acquired by the camera in front of the vehicle The composite line of the front upper viewpoint image that is the viewpoint image matches the front line of the area occupied by the work vehicle of the display unit, and the upper right viewpoint image, the upper left viewpoint image, and the front upper viewpoint The right upper viewpoint image, the left upper viewpoint image, and the front upper viewpoint image are generated so as to have the same image scale, and the second upper viewpoint overhead image is acquired by the camera on the right side of the vehicle. A right upper viewpoint image that is an upper viewpoint image based on the captured image, an upper left viewpoint image that is an upper viewpoint image based on an image acquired by the camera on the left side of the vehicle, and a rear view of the vehicle A composite line of a rear upper viewpoint image that is an upper viewpoint image based on an image acquired by a camera matches a rear line of the area occupied by the work vehicle of the display unit, and the upper right viewpoint image, For a work vehicle surrounding monitoring device that is generated by combining the upper right viewpoint image, the upper left viewpoint image, and the rear upper viewpoint image so that the scales of the upper left viewpoint image and the rear upper viewpoint image are the same . It is a program.

  ADVANTAGE OF THE INVENTION According to this invention, the surroundings monitoring apparatus which is easy to confirm the circumference | surroundings and the front of a vehicle can be provided.

FIG. 1 is a side view of a work machine equipped with a periphery monitoring system. FIG. 2 is a plan view of the work machine. FIG. 3 is a block diagram showing the overall configuration of the surrounding monitoring system. FIG. 4 is a diagram for explaining image conversion in the periphery monitoring device. FIG. 5 is a diagram showing a front viewpoint image 600. FIG. 6 is a diagram showing an upper viewpoint image 700 used for synthesis of the front confirmation image. FIG. 7 is a diagram showing a two-viewpoint image 800. FIG. 8 is a diagram showing a front confirmation image 900. FIG. 9 is a diagram showing a rear viewpoint image 1600. FIG. 10 is a diagram showing an upper viewpoint image 1700. FIG. 11 is a diagram showing a two-viewpoint image 1800. FIG. 12 is a diagram showing a backward confirmation image 1900. FIG. 13 is a flowchart for explaining the image processing flow. FIG. 14 is a diagram illustrating an example of a compacting operation using the tire roller 400. FIG. 15 is a diagram showing a surrounding monitoring image in the situation shown in FIG.

  Hereinafter, embodiments for carrying out the present invention will be described with reference to the drawings. 1 and 2 are views showing the appearance of a work machine equipped with a periphery monitoring system. FIG. 1 is a side view and FIG. 2 is a plan view. In the present embodiment, a tire roller, which is a road machine for the purpose of compacting a road, will be described as an example of a work machine. However, the surrounding monitoring device of the present invention is not limited to a tire roller, and a road roller, a wheel loader, and the like It can also be applied to other work machines.

  As shown in FIG. 1, the tire roller 400 includes a tire roller main body 401, a drive shaft tire 402, and an idle shaft tire 403. The tire roller body 401 is provided with a driver seat 404, a handle 405, and a canopy 406 that covers the driver seat 404. The tire roller body 401 is provided with a surrounding monitoring device 200 and a display device 265 that constitute a surrounding monitoring system. The display device 265 is disposed in front of the driver's seat 404 and on the side of the handle 405, and is connected to the surroundings monitoring device 200 via a cable. An operator seated on the driver's seat 404 can drive the tire roller 400 while monitoring the image displayed on the display device 265.

  As shown in FIG. 2, a front camera 205, a rear camera 210, a right camera 215, and a left camera 220 are provided at the front, rear, right side, and left side of the tire roller body 401. These cameras 205, 210, 215, and 220 are connected to the surroundings monitoring apparatus 200 shown in FIG.

  The cameras 205, 210, 215, and 220 are composed of, for example, a wide-angle video camera provided with an imaging element such as a CCD or CMOS excellent in durability and weather resistance and a wide-angle lens. The front camera 205 is arranged so that the angle of view is approximately 180 degrees and is looked down obliquely, and the photographing range 510 covers the front of the tire roller 400. The rear camera 210 is also arranged so as to look down obliquely with an angle of view of about 180 degrees, and the photographing range 520 covers the rear of the tire roller 400. The right camera 215 is also arranged so as to look down obliquely with an angle of view of about 180 degrees, and the photographing range 530 covers the right side of the tire roller 400. The left camera 220 is also arranged so that the angle of view is approximately 180 degrees and is looked down obliquely, and the shooting range 540 covers the left side of the tire roller 400.

  The front camera 205 and the right camera 215 are arranged so that the shooting range 510 and the shooting range 530 overlap each other. In addition, the front camera 205 and the left camera 220 are arranged so that the shooting range 510 and the shooting range 540 overlap. Similarly, the rear camera 210 and the right camera 215 or the left camera 220 are arranged so that the shooting range 520 and the shooting range 530 or the shooting range 540 overlap.

  FIG. 3 is a block diagram showing the overall configuration of the surrounding monitoring system. The surrounding monitoring system includes a front camera 205, a rear camera 210, a right camera 215, a left camera 220, a surrounding monitoring device 200, and a display device 265. The surrounding monitoring apparatus 200 includes a video acquisition unit 225, a traveling direction viewpoint image generation unit 230, a display image generation unit 235, a switching determination unit 240, a buffer 250, an upper viewpoint image generation unit 255, a two viewpoint image synthesis unit 260, and a control unit 270. And an input / output interface 245. The display device 265 includes a display unit 2650 and an input unit 2651 (for example, a liquid crystal display device is used).

  A composite signal such as NTSC of the original image is input from each of the cameras 205, 210, 215, and 220 to the video acquisition unit 225, and the video acquisition unit 225 performs A / D conversion on each composite signal and decodes it to an RGB signal. . The video data processed by the video acquisition unit 225 is temporarily stored in the buffer 250.

  Based on the video data stored in the buffer 250, the upper viewpoint image generation unit 255 creates an image (hereinafter referred to as an upper viewpoint image) that looks down from the viewpoint above the vehicle. The traveling direction viewpoint image generation unit 230 generates an image related to the viewpoint toward the traveling direction of the tire roller 400. That is, when the vehicle moves forward, an image of the vehicle front viewed from the viewpoint toward the front of the vehicle (hereinafter referred to as a front viewpoint image) is generated, and when the vehicle reverses, an image of the rear of the vehicle viewed from the viewpoint toward the rear of the vehicle (hereinafter referred to as the rear Called a viewpoint image).

  The two-viewpoint image synthesis unit 260 synthesizes the upper viewpoint image generated by the upper viewpoint image generation unit 255 and the traveling direction viewpoint image generated by the traveling direction viewpoint image generation unit 230 into one image. When the above-described front viewpoint image and upper viewpoint image are combined into a two viewpoint image, an upper viewpoint image based on the images of the front camera 205, the right camera 215, and the left camera 220 is used as the upper viewpoint image. The two-viewpoint image after the synthesis process in this case is referred to as a front confirmation image. On the other hand, when the rear viewpoint image and the upper viewpoint image are combined, the upper viewpoint image based on the images of the rear camera 210, the right camera 215, and the left camera 220 is used as the upper viewpoint image. The two-viewpoint image after the synthesis process in this case is referred to as a rear confirmation image.

  The display image generation unit 235 generates a display image by synthesizing a display (such as an icon) indicating the tire roller 400 with the two viewpoint images (front confirmation image and rear confirmation image) generated by the two viewpoint image combination unit 260. . Determination information for determining whether to synthesize the front confirmation image or the rear confirmation image is input to the switching determination unit 240. Based on the input determination information, the switching determination unit 240 sends a command indicating which of the front confirmation image and the rear confirmation image should be combined to the two-viewpoint image combination unit 260. When a command to synthesize the front confirmation image is input, the two-viewpoint image synthesis unit 260 generates a two-viewpoint image for the front confirmation image and receives a command to synthesize the rear confirmation image. A 2-viewpoint image for the backward confirmation image is generated.

  The determination information input to the switching determination unit 240 includes, for example, position information (front / back / neutral) of a shift lever that switches between forward and backward travel of the vehicle. Further, an image changeover switch may be provided in the display device 265, and operation information of the image changeover switch may be used. The image changeover switch may be a changeover button displayed on the display unit 2650, or a hardware button such as a toggle switch or button.

  The display image (front confirmation image or rear confirmation image) generated by the display image generation unit 235 is displayed on the display unit 2650 of the display device 265. The input unit 2651 includes the above-described image switching switch, a switch for performing an input operation related to image display, and the like. The control unit 270 controls the entire surroundings monitoring apparatus 200 and executes a program related to image generation processing described later. The image generation processing program is stored in a memory (for example, ROM) provided in the control unit 270.

  4-7 is a figure explaining the production | generation method of a front confirmation image. FIG. 4A is a diagram illustrating an image captured by the front camera 205. Specifically, an image is shown when it is assumed that a flat road surface on which the lattice lines 32a and 32b are drawn is imaged. Since each of the cameras 205, 210, 215, and 220 is photographed using a wide-angle lens having a left and right angle of view of about 180 °, generally, the grid lines 32a and 32b in the photographed image 31 are enlarged at the center. And the peripheral part is distorted so as to be reduced.

  FIG. 4B shows an image 33 after the lens distortion correction processing is performed on the image 31 of FIG. In the image 33 after the correction process, the grid lines 32 a and 32 b in FIG. 4A are corrected to a shape according to the perspective method based on the viewpoint of the front camera 205. As a result, the lattice lines 32a and 32b are converted into linear lattice lines 34a and 34b. The image 33 is an image according to the looking down angle of the front camera 205.

  Furthermore, when a viewpoint conversion process is performed on the image 33 after the lens distortion correction process, the viewpoint is shifted upward (that is, when looking down from the viewpoint above the vehicle), an image 35 as shown in FIG. (Upper viewpoint image). The lattice lines 34a and 34b in FIG. 4B are converted into lattice lines 36a and 36b orthogonal to the top, bottom, left, and right in FIG. 4C. Here, the image of the rectangular region 37 shown in FIG. 4B is converted into the image of the trapezoidal region 39 of FIG. Further, the image of the trapezoidal region 38 in FIG. 4B is converted into the image of the rectangular region 40 in FIG.

  For the images acquired by the right camera 215, the left camera 220, and the rear camera 210, images similar to the images 31, 33, and 35 are generated by performing conversion as shown in FIG.

  The conversion from the image 31 in FIG. 4A to the image 33 in FIG. 4B and the conversion from the image 33 in FIG. 4B to the image 35 in FIG. By pixel coordinate conversion using a dedicated pixel conversion table (pixel conversion table describing the correspondence between the addresses of each pixel constituting the image before conversion and the addresses of each pixel after conversion) stored in the memory Is called. Also, when converting from the image in FIG. 4A to the image in FIG. 4C without passing through the image in FIG. 4B, image conversion is performed by the same pixel coordinate conversion.

  Since an image taken with a wide-angle camera has a large image distortion, when a straight line is taken, the straight line becomes a curved image. However, in the present embodiment, a lens distortion correction process is performed when converting the image 31 in FIG. 4A to the image 33 in FIG. ) Image 33 is also displayed as a linear image. In FIG. 4B, the lattice lines 34a and 34b are straight lines.

  3 generates a forward viewpoint image 600 as a traveling direction viewpoint image as shown in FIG. 5 based on the image data of the region 37 in FIG. 4B. Of course, the front viewpoint image 600 may be directly generated from a part of the image 33 as shown in FIG. 4A (an image corresponding to the image in the region 37 in FIG. 4B).

  The image of the region 37 in FIG. 4B described above is an image corresponding to the looking-down angle of the traveling direction viewpoint, but the looking-down angle in the front viewpoint image 600 is a predetermined looking-down angle for displaying on the display unit 2650. It is said that. If this looking-down angle is the same as the looking-down angle of the front camera 205, the image of the region 37 in FIG. 4B and the front viewpoint image 600 in FIG. 5 are the same image. The straight line (grid line) 610 of the front viewpoint image 600 corresponds to the grid line 32a in FIG. 4A, the grid line 34a in FIG. 4B, and the grid line 36a in FIG. In the front viewpoint image 600 of FIG. 5, the grid lines corresponding to the grid lines 36 b of FIG. 4C are not shown.

  The upper viewpoint image generation unit 255 in FIG. 3 performs processing for converting to the viewpoint from above, similar to FIG. 4C, based on the video data acquired from the front camera 205, the right camera 215, and the left camera 220. Thus, an upper viewpoint image of the front portion of the tire roller 400 is generated. The upper viewpoint image of the front part is used when a front confirmation image is synthesized. Similarly, based on the video data acquired from the rear camera 210, the right camera 215, and the left camera 220, an upper viewpoint image of the rear portion of the tire roller 400 is generated. The upper viewpoint image in the rear part is used when a rear confirmation image is synthesized.

  FIG. 6 is a diagram showing an upper viewpoint image 700 used for synthesis of the front confirmation image. Of the upper viewpoint image 700, the image of the area indicated by reference numeral 710 is the upper viewpoint image based on the video data of the front camera 205, and corresponds to the image of the rectangular area 40 shown in FIG. Further, the image of the area indicated by reference numeral 720 is an upper viewpoint image based on the video data of the right camera 215, and the image of the area indicated by reference numeral 730 is an upper viewpoint image based on the video data of the left camera 220. A region 750 having no grid line between the image 720 and the image 730 is a region occupied by the tire roller 400. Images 720 and 730 correspond to the upper viewpoint image of the front half in the upper viewpoint image of the side region of the vehicle body of the tire roller 400.

  When combining these three images, the composite line 740 of the image 720 of the right camera 215 and the image 730 of the left camera 220 and the image 710 of the front camera 205 corresponds to the front line of the region 750 (corresponding to the front surface of the tire roller 400). The composition processing is performed so that the line is substantially parallel to the line and coincides with the line. Further, the images are synthesized so that the intervals of the grid lines 710a, 710b, 720a, 720b, 730a, and 730b in the images 710, 720, and 730 are the same so that the scales of the images are the same.

  3 synthesizes the front viewpoint image 600 shown in FIG. 5 and the upper viewpoint image 700 shown in FIG. 6 to generate a two viewpoint image 800 as shown in FIG. The In this composition, image composition is performed so that the grid line 610 of the front viewpoint image 600 and the grid line 710a of the upper viewpoint image 700 are connected. As described above, when combining the images 600, 710, 720, and 730, the images are connected before and after the boundary.

  In the display image generation unit 235 of FIG. 3, a front confirmation image 900 as shown in FIG. 8 is generated by synthesizing the vehicle icon 910 indicating the area of the tire roller 400 with the two viewpoint image 800. A vehicle icon 910 in the front confirmation image 900 corresponds to an upper viewpoint image from the center of the tire roller 400 to the front end. The vehicle icon 910 is displayed adjacent to the center of the lower side of the display screen of the display unit 2650, and the two viewpoint images 800 around the tire roller 400 are displayed on the display unit 2650 with reference to the vehicle icon 910.

  In the two-viewpoint image 800, the grid lines 610, 710a, and 720a are continuous, and the grid lines 710a and 720a are parallel to the longitudinal axis of the vehicle icon 910. Further, the grid line 610 has a shape according to the perspective from the viewpoint of the front camera 205, that is, a straight line inclined toward the center of the upper side of the two-viewpoint image 800.

  As described above, in the present embodiment, when the two-viewpoint image 800 of FIG. 7 is generated based on the video data captured by the camera, the lens distortion correction process is performed and the images 600 and 700 to 730 are combined. In this case, image synthesis is performed so that the grid lines of the images are straight lines. As a result, it is easy to determine whether the road shoulder in front of the vehicle is a straight line or a curve, and it is easy to drive along the road shoulder. Further, by displaying the front viewpoint image 600, an image ahead of the vehicle is provided far away, and the driver can check the state of the traveling direction.

  9-12 is a figure explaining a back confirmation image, and is a figure equivalent to FIGS. 5-8 regarding the front confirmation image 900 mentioned above. FIG. 9 is a diagram illustrating a rear viewpoint image 1600 as a traveling direction viewpoint image based on the video data of the rear camera 210. In the rear viewpoint image 1600, the upper side image is an image close to the vehicle, and the lower side image is a distant image. Therefore, the lattice line 1610 is a straight line inclined toward the lower side center.

  FIG. 10 shows an upper viewpoint image 1700 when a backward confirmation image is synthesized. In the upper viewpoint image 1700, the upper viewpoint image (image 1730) based on the video data of the left camera 220 and the upper side based on the video data of the right camera 215 are located above the upper viewpoint image (image 1710) based on the video data of the rear camera 210. A viewpoint image (image 1720) is arranged. Images 1720 and 1730 correspond to an upper viewpoint image in the rear half of the upper viewpoint image of the side region of the vehicle body of the tire roller 400. A composite line 1740 of the images 1720 and 1730 and the image 1710 is formed to be parallel to the rear surface of the tire roller 400 and coincide with the rear surface of the tire roller 400. Area 1750 is an area where a vehicle icon is displayed.

  FIG. 11 shows a two-viewpoint image 1800 when a rear confirmation image is synthesized. In the two-viewpoint image 1800, the upper viewpoint image and the rear viewpoint image 1600 based on the video data of the rear camera 210 are connected, that is, the rear viewpoint image 1600 is connected to the lower side of the upper viewpoint image 1700. Done. FIG. 12 is a view showing a rear confirmation image 1900 obtained by combining the vehicle icon 1910 with the two viewpoint image 1800 of FIG. A vehicle icon 1910 in the rear confirmation image 1900 corresponds to an upper viewpoint image from the center of the tire roller 400 to the rear end.

  Next, an image processing flow processed by the control unit 270 of FIG. 3 will be described with reference to FIG. In step S300, activation processing for confirming the activation state of the surrounding monitoring system and performing error diagnosis is performed. In step S 305, the video acquisition unit 225 digitizes the video from the front camera 205, the rear camera 210, the right camera 215, and the left camera 220, and further temporarily stores the digitized video in the buffer 250.

  In step S310, by determining whether the determination information indicating the monitoring direction managed by the switching determination unit 240 is forward or backward, the forward confirmation image 900 (see FIG. 8) or the backward confirmation image 1900 (see FIG. 12). Decide which one to generate. If the monitoring direction is forward, the process proceeds to step S315. If the monitoring direction is backward, the process proceeds to step S325.

  In step S315, the forward viewpoint image 600 of FIG. 5 is generated by performing processing for converting the video acquired by the front camera 205 into a viewpoint in the traveling direction by the traveling direction viewpoint image generation unit 230. In step S320, the upper viewpoint image generation unit 255 performs processing for converting the images acquired from the right camera 215, the left camera 220, and the front camera 205 into the viewpoint from above, so that the upper viewpoint of the front portion of the tire roller 400 is processed. An image 700 (see FIG. 6) is generated.

  On the other hand, when the process proceeds from step S310 to step S325, the rear viewpoint image 1600 is obtained by performing processing of converting the video acquired by the rear camera 210 into a viewpoint in the traveling direction in the traveling direction viewpoint image generation unit 230 in step S325. (See FIG. 9).

  In step S330, the upper viewpoint image generation unit 255 performs processing for converting the video acquired from the right camera 215, the left camera 220, and the rear camera 210 into a viewpoint from above, so that the upper viewpoint of the rear portion of the tire roller 400 is processed. Generate an image. If the process of step S330 is completed, the process proceeds to step S335.

  In step S335, the two-viewpoint image synthesizing unit 260 synthesizes the traveling direction viewpoint image (front viewpoint image 600 or rear viewpoint image 1600) generated in step S315 or step S325 and the upper viewpoint image created in step S320 or step S330. By processing, a two-viewpoint image 800 or 1800 (see FIGS. 7 and 11) is generated.

  In step S340, the two-viewpoint images 800 and 1800 generated in step S335 and the vehicle icons 910 and 1910 (see FIGS. 8 and 12) of the tire roller 400 are combined by the display image generation unit 235, and the front confirmation image 900 (FIG. 8) or a backward confirmation image 1900 (see FIG. 12).

  In step S345, the front confirmation image 900 (see FIG. 8) or the rear confirmation image 1900 (see FIG. 12) created in step S340 is displayed on the display unit 2650 in FIG.

  As described above, the control unit 270 displays a surrounding monitoring device program for displaying a monitoring image around the vehicle based on the images around the vehicle captured by the plurality of cameras 205 to 220 provided on the vehicle. Run. That is, the program for the surroundings monitoring apparatus generates a front viewpoint image 600 or a rear viewpoint image 1600 based on an image acquired by the front camera 205 or the rear camera 210, an image acquired by the front camera 205 or the rear camera 210, Processing for generating an upper viewpoint image 700 or 1700 corresponding to the front viewpoint image 600 or the rear viewpoint image 1600 based on the left side image acquired by the left camera 220 and the right side image acquired by the right camera 215, and the upper viewpoint image 700 or 1700, a process of generating a monitoring image by combining the front viewpoint image 600 or the rear viewpoint image 1600 corresponding to the upper viewpoint image 700 or 1700, and a process of displaying the monitoring image on the display unit 2650 by a computer. To be executed.

  FIG. 14 is a diagram illustrating an example of a compacting operation using the tire roller 400, and illustrates a case where the compacting operation of the road 420 is performed. A driver (not shown) of the tire roller 400 performs driving operations such as steering and acceleration / deceleration of the tire roller 400 at the driver seat 410. In the compaction work of the road 420, the compaction work by the tire roller 400 is performed, and the worker 110 works around the tire roller 400 in order to collect spilled asphalt. The driver compacts the road surface by moving the tire roller 400 forward and backward along the shoulder 430 of the road.

  FIG. 15 shows a surrounding monitoring image in the situation shown in FIG. 14, that is, a front confirmation image 100 (corresponding to the above-described front confirmation image 900) displayed on the display unit 2650. The front confirmation image 100 is obtained by combining the upper viewpoint image 125 on the lower side of the front viewpoint image 120 and further combining the icon 105 indicating the region of the tire roller 400. In the front viewpoint image 120, a worker 110 and left and right road shoulders 115 are displayed. In the upper viewpoint image 125, road shoulders 115 are displayed on the left and right of the icon 105.

  Thus, by displaying the upper viewpoint image 125 as seen from above the vehicle, it is possible to provide an image in which the positional relationship between the vehicle and the object (road shoulder or the like) located around the vehicle is easily understood. Furthermore, by displaying the front viewpoint image 120 in addition to the upper viewpoint image 125, not only the situation around the vehicle but also the worker 110 at a position slightly away from the vehicle can be simultaneously grasped on the display screen. The road shoulder 430 shown in FIG. 14 extends linearly in the vehicle front-rear direction, but the road shoulder 115 in the front confirmation image 100 is also displayed as a single road shoulder 115 that spans the front viewpoint image 120 and the upper viewpoint image 125. . Further, the road shoulder 115 in the front viewpoint image 120 is displayed in a straight line without distortion. Therefore, it becomes easy to determine whether the road shoulder in front of the vehicle is a straight line or a curve, and driving along the road shoulder becomes easy.

  In the above-described embodiment, as described in the flowchart of FIG. 13, the front confirmation image 900 or the rear confirmation image 1900 is selectively generated depending on whether the determination information input to the switching determination unit 240 is front or rear. Although the configuration is adopted, a configuration in which only the front confirmation image 900 is synthesized and displayed, or a configuration in which only the rear confirmation image 1900 is synthesized and displayed may be employed.

As described above, the surroundings monitoring apparatus according to the present embodiment has the following features.
(A) The traveling direction viewpoint image generation unit 230, the upper viewpoint image generation unit 255, the two viewpoint image synthesis unit 260, and the display image generation unit 235 corresponding to the image generation unit generate at least one of the front confirmation image and the rear confirmation image. To do. The front confirmation image 900 includes a front viewpoint image 600 generated based on the video acquired by the front camera 205, and an upper viewpoint image generated based on the images acquired by the front camera 205, the left camera 220, and the right camera 215. 700. The rear confirmation image 1900 includes a rear viewpoint image 1600 generated based on the video acquired by the rear camera 210 and an upper viewpoint image generated based on each video acquired by the rear camera 210, the left camera 220, and the right camera 215. 1700. Thus, in addition to the upper viewpoint images 700 and 1700 around the vehicle, the front viewpoint image 600 and the rear viewpoint image 1600 are combined with the front confirmation image 900 and the rear confirmation image 1900. Therefore, the driver can view the traveling direction and the left-right direction of the vehicle simultaneously from the front confirmation image 900 and the rear confirmation image 1900, and there is no need to move the viewpoint for confirming the front situation and the surrounding situation as in the past. Thus, it is possible to simultaneously recognize information such as obstacles in the traveling direction and information such as straightness and continuity of the shoulder.

  In the above-described embodiment, the forward confirmation image 900 in which the vehicle icon 910 is combined is displayed on the display unit 2650 as a monitoring image. However, an area 750 in which the vehicle icon 910 is arranged like the two-viewpoint image 800 is displayed. A blank image may be displayed on the display unit 2650 as a monitoring image.

(B) Further, as in the front confirmation image 900 or the rear confirmation image 1900, the vehicle icons 910 and 1910 indicating the work machine area are combined and displayed in a predetermined display area (areas 750 and 1750), so that Alternatively, it is possible to intuitively grasp the distance from the obstacle.

(C) Also, as shown in FIG. 6, the upper viewpoint image 710 and the upper viewpoint image (720, 730) based on the video data of the front camera 205 pass through the vehicle front of the vehicle icon 910 and are parallel to the vehicle front. Since the images are synthesized so as to be connected along the synthesis line 740, distortion of the image near the road shoulder on the synthesized image can be suppressed, and the road shoulder can be easily seen. When connected along the parallel synthesis line 740, the image data of the camera closer to the subject is used to generate an image near the connection region, and a display image with higher resolution can be obtained. As a result, the visibility of the shoulder is improved.

(D) A switching determination unit as an image selection command input unit to which an image selection command (the above-described determination information) for selecting one of the front confirmation image 900 and the rear confirmation image 1900 is displayed as a display image displayed on the display unit 2650 240 and the display image generation unit 235 may generate the front confirmation image 900 or the rear confirmation image 1900 based on the determination information. By adopting such a configuration, for example, the front confirmation image 900 can be displayed on the display unit 2650 when the vehicle moves forward, and the rear confirmation image 1900 can be displayed when the vehicle moves backward, which improves vehicle operability. It becomes possible to plan.

  In addition, the above description is an example to the last, and this invention is not limited to the said embodiment at all unless the characteristic of this invention is impaired. For example, in the above-described embodiment, the front confirmation image 900 or the rear confirmation image 1900 is generated according to the determination information, and the generated image is displayed on the display unit 2650. It is good also as a structure which produces | generates both the confirmation image 900 and the back confirmation image 1900, and displays either on the display part 2650 according to determination information.

  115, 430 ... Road shoulder, 200 ... Ambient monitoring device, 205 ... Front camera, 210 ... Rear camera, 215 ... Right camera, 220 ... Left camera, 225 ... Video acquisition unit, 230 ... Traveling direction viewpoint image generation unit, 235 ... Display Image generation unit, 240 ... switching determination unit, 245 ... input / output interface, 255 ... upper viewpoint image generation unit, 260 ... two viewpoint image synthesis unit, 265 ... display device, 400 ... tire roller, 600, 1600 ... front viewpoint image, 700, 1700 ... upper viewpoint image, 740, 1740 ... composite line, 800, 1800 ... two viewpoint images, 900 ... front confirmation image, 910, 1910 ... vehicle icon, 1900 ... rear viewpoint image, 2650 ... display section

Claims (3)

Based on the image of the vehicle periphery captured by a plurality of cameras provided in the vehicle, an image generator for generating a rear view picture image for confirming the front check image and the rear of the vehicle for checking the front of the vehicle ,
An image selection command input unit for inputting an image selection command for selecting one of the generated front confirmation image and the rear confirmation image;
An output unit that outputs the image selected by the image selection command input unit to a display unit,
The front confirmation image is generated based on a front viewpoint image generated based on an image acquired by a camera in front of the vehicle, and images acquired by a camera in front of the vehicle, a camera on the left side of the vehicle, and a camera on the right side of the vehicle. The front viewpoint image and the first image along the line parallel to the vehicle front surface of the vehicle icon imitating the work vehicle in the front confirmation image. 1 is synthesized so that the overhead view image is connected,
The rear confirmation image is generated based on a rear viewpoint image generated based on an image acquired by a camera behind the vehicle, and each image acquired by a camera behind the vehicle, a camera on the left side of the vehicle, and a camera on the right side of the vehicle. The rear viewpoint image and the second upper viewpoint overhead view along a line that passes through the vehicle rear surface of the vehicle icon and is parallel to the vehicle rear surface in the rear confirmation image. It is synthesized so that the image is connected,
The first upper viewpoint overhead image is an upper viewpoint image that is an upper viewpoint image based on an image acquired by a camera on the right side of the vehicle and an upper left image that is an upper viewpoint image based on an image acquired by the camera on the left side of the vehicle. The composite line of the viewpoint image and the front upper viewpoint image that is the upper viewpoint image based on the image acquired by the camera in front of the vehicle matches the front line of the area occupied by the work vehicle of the display unit, and Generated by combining the upper right viewpoint image, the upper left viewpoint image, and the upper front viewpoint image so that the scales of the upper right viewpoint image, the upper left viewpoint image, and the front upper viewpoint image are the same,
The second upper viewpoint overhead image is an upper right viewpoint image that is an upper viewpoint image based on an image acquired by a camera on the right side of the vehicle and an upper left image that is an upper viewpoint image based on an image acquired by the camera on the left side of the vehicle. The composite line of the viewpoint image and the rear upper viewpoint image that is the upper viewpoint image based on the image acquired by the camera behind the vehicle coincides with the rear line of the area occupied by the work vehicle of the display unit, and The upper right viewpoint image, the upper left viewpoint image, and the rear upper viewpoint image are generated by synthesizing the upper right viewpoint image, the upper left viewpoint image, and the rear upper viewpoint image,
The image selection instruction input section, Ru image switching switch der accept the image selection command from the operator, surroundings monitoring apparatus for a working vehicle. In the surrounding monitoring apparatus of the work vehicle of Claim 1,
The image generation unit is configured such that the vehicle icon is displayed at a predetermined display area of the display unit synthesizes the forward confirmation image or the rear view image and the vehicle icon, surrounding monitoring apparatus for a working vehicle. An image generation function for generating a front confirmation image for confirming the front of the vehicle and a rear confirmation image for confirming the rear of the vehicle , based on images around the vehicle imaged by a plurality of cameras provided in the vehicle ;
An image selection function for receiving an image selection command for selecting one of the generated front confirmation image and the rear confirmation image via an image changeover switch for receiving a command from an operator;
A work vehicle surrounding monitoring device program for causing a computer to realize an output function of outputting an image accepted by the image selection function to a display unit ,
The front confirmation image is generated based on a front viewpoint image generated based on an image acquired by a camera in front of the vehicle, and images acquired by a camera in front of the vehicle, a camera on the left side of the vehicle, and a camera on the right side of the vehicle. And the first upper viewpoint overhead image and the front viewpoint image along the line parallel to the front of the vehicle passing through the front of the vehicle icon of the vehicle in the forward confirmation image and imitating the work vehicle. The first upper viewpoint overhead image is synthesized so as to be connected,
The rear confirmation image is generated based on a rear viewpoint image generated based on an image acquired by a camera behind the vehicle, and each image acquired by a camera behind the vehicle, a camera on the left side of the vehicle, and a camera on the right side of the vehicle. The rear viewpoint image and the second upper viewpoint overhead view along a line that passes through the vehicle rear surface of the vehicle icon and is parallel to the vehicle rear surface in the rear confirmation image. It is synthesized so that the image is connected,
The first upper viewpoint overhead image is an upper viewpoint image that is an upper viewpoint image based on an image acquired by a camera on the right side of the vehicle and an upper left image that is an upper viewpoint image based on an image acquired by the camera on the left side of the vehicle. The composite line of the viewpoint image and the front upper viewpoint image that is the upper viewpoint image based on the image acquired by the camera in front of the vehicle matches the front line of the area occupied by the work vehicle of the display unit, and Generated by combining the upper right viewpoint image, the upper left viewpoint image, and the upper front viewpoint image so that the scales of the upper right viewpoint image, the upper left viewpoint image, and the front upper viewpoint image are the same,
The second upper viewpoint overhead image is an upper right viewpoint image that is an upper viewpoint image based on an image acquired by a camera on the right side of the vehicle and an upper left image that is an upper viewpoint image based on an image acquired by the camera on the left side of the vehicle. The composite line of the viewpoint image and the rear upper viewpoint image that is the upper viewpoint image based on the image acquired by the camera behind the vehicle coincides with the rear line of the area occupied by the work vehicle of the display unit, and The upper right viewpoint image, the upper left viewpoint image, and the rear upper viewpoint image are generated by combining the upper right viewpoint image, the upper left viewpoint image, and the rear upper viewpoint image so that the scales thereof are the same. A program for monitoring the surroundings of work vehicles

Images