JP6418604B2 - Working machine and control method thereof - Google Patents

Description

  The present invention relates to a working machine that travels along a cultivated land in which ridges in which crops are planted are formed in a row, and a control method thereof.

  2. Description of the Related Art Conventionally, for example, a working machine has been proposed that travels along a cultivated land in which vines in which tea trees are planted as crops are formed in a row (for example, see Patent Document 1 below).

  Specifically, in Patent Document 1 below, the contact pressure of the sensing plate on the side surface of the teacup is measured by a pressure sensor provided on the sensing plate so that the contact pressure on both sides of the teacup becomes equal. Describes a technique for performing automatic traveling control.

JP 2011-234623 A

  However, in the technique of Patent Document 1, since it is necessary to bring the sensing plate into contact with the side surface of the teacup in order to perform automatic traveling control, the teacup (specifically, the side surface of the teacup) is destroyed. There was a concern.

  This invention is made | formed in view of such a problem, and it aims at providing the mechanism which can perform automatic driving | running | working control, without destroying the basket | plant in which the crop is planted.

The working machine of the present invention is a working machine that travels along the cultivated land in which the ridges in which crops are planted are formed in a row, and the shooting direction is set in the traveling direction of the operative machine. In this state, the area of the cultivated land including the path between the adjacent ridges and the ridge is photographed from obliquely above, and an imaging means for obtaining an original image of the area, and the original using the binarization threshold. A binarization processing unit that binarizes the image to generate a binary image; a plane projection conversion processing unit that generates a plane projection conversion binary image by performing a plane projection conversion process on the binary image; and the plane Image rotation processing means for generating a plurality of rotation images corresponding to the rotation angles of the plurality of rotation angles by performing image rotation processing for rotating the projective transformation binary image at a plurality of rotation angles, and the plurality of rotation images For each rotated image in, at each horizontal position Calculating the number of pixels less than the binarization threshold among the pixels in the vertical direction orthogonal to the horizontal direction and obtaining the passage existence probability distribution based on the number of pixels less than the binarization threshold A probability distribution acquisition means; an optimum rotation image extraction means for extracting a rotation image having a maximum maximum value in the existence probability distribution of the passage among the plurality of rotation images as an optimum rotation image; and the rotation angle in the optimum rotation image. And an angle calculating means for calculating an angle of the work implement in the traveling direction with respect to the rod.
Another aspect of the work machine according to the present invention is a work machine that travels along a cultivated land in which ridges in which crops are planted are formed in a row, and can photograph the traveling direction of the work machine. The cultivated land that is arranged at the upper part of the front part of the work implement and includes at least two passages between the adjacent ridges and the ridge in a state in which the photographing direction is set in the traveling direction of the work implement. Photographing means for photographing the area from above and obtaining an original image of the area; and binarization processing means for generating a binary image by binarizing the original image using a binarization threshold value; A plane projection conversion processing means for generating a plane projection conversion binary image by performing a plane projection conversion process on the binary image, and an image rotation process for rotating the plane projection conversion binary image at a plurality of rotation angles, A plurality of rotation images corresponding to each rotation angle in the plurality of rotation angles. And the number of pixels less than the binarization threshold among the pixels in the vertical direction orthogonal to the horizontal direction at each position in the horizontal direction for each rotated image in the plurality of rotated images. And the existence probability distribution acquisition means for acquiring the existence probability distribution of the passage based on the number of pixels less than the binarization threshold, and the maximum value in the existence probability distribution of the passage among the plurality of rotated images is An optimum rotation image extraction means for extracting the largest rotation image as an optimum rotation image, a center position in the horizontal direction in the optimum rotation image, a position where the two passages exist in the optimum rotation image, and the work implement And a positional deviation amount calculating means for calculating a positional deviation amount from the center position of the heel of the work machine based on the position of the photographing means.
The present invention also includes the above-described work machine control method.

  According to the present invention, it is possible to perform automatic traveling control without destroying the basket in which the crop is planted.

It is a figure which shows typically a mode that the working machine which concerns on embodiment of this invention drive | works a cultivated land. It is a figure which shows an example of a function structure of the working machine which concerns on embodiment of this invention. It is a figure which shows an example of the various images obtained by the image process part of FIG. It is a figure for demonstrating embodiment of this invention and demonstrating an example of the morphological filtering performed in the binarization process part of FIG. It is a figure which shows embodiment of this invention and shows an example of the presence probability distribution of the path | route in a certain rotation image acquired by the presence probability distribution acquisition part of FIG. FIG. 3 is a diagram illustrating an embodiment of the present invention and explaining a positional deviation amount calculation process by a positional deviation amount calculation unit in FIG. 2. It is a figure which shows embodiment of this invention and shows an example of the screen display by the display part of FIG. It is a flowchart which shows an example of the process sequence in the control method of the working machine which concerns on embodiment of this invention.

  Hereinafter, embodiments (embodiments) for carrying out the present invention will be described with reference to the drawings. In addition, in embodiment of this invention shown below, the example which assumed the working machine which drive | works along the said cultivation field where the basket (tea bowl) in which the tea tree was planted as a crop along the said basket is demonstrated is demonstrated. However, the present invention is not limited to this form. A work machine that travels along a cultivated land in which cocoons in which crops other than tea trees (for example, crops such as sweet potatoes, leeks, cabbage, etc.) are planted in rows is also applicable to the present invention. It is.

  FIG. 1 is a diagram schematically illustrating a state in which a work machine 100 according to an embodiment of the present invention travels on a cultivated land 200. Specifically, Fig.1 (a) is the figure which looked at the working machine 100 and the cultivated land 200 from the upper direction. Moreover, FIG.1 (b) is the figure which looked at the working machine 100 and the cultivated land 200 from the side.

  As shown to Fig.1 (a), the cultivated land 200 has the row | line | column (it is set as a teacup in this embodiment) 210 in which the crop (it is set as a tea tree in this embodiment) is planted in a line form. . In addition, a passage 220 through which a traveling mechanism (caterpillar, tire, or the like) of the work machine 100 passes is formed between the adjacent saddles 210 and 210.

  The work machine 100 travels on the cultivated land 200 along the ridge 210. As shown in FIG. 1A and FIG. 1B, the working machine 100 includes a cultivated land including a ridge 210 and a passage 220 with a shooting direction set in the traveling direction 101 of the working machine 100. An imaging unit 110 is provided that captures 200 areas obliquely from above and obtains an original image of the area. At this time, in the present embodiment, the imaging unit 110 is disposed at the center position in the orthogonal direction 102 of the work machine 100, which is orthogonal to the traveling direction 101 of the work machine 100, as illustrated in FIG. . In the present invention, the photographing unit 110 is not necessarily arranged at the center position in the orthogonal direction 102 of the work machine 100. For example, the work machine 100 capable of photographing the traveling direction 101 of the work machine 100. The form arrange | positioned at the upper part of the front part of may be sufficient.

  Moreover, in this embodiment, the imaging | photography part 110 image | photographs the area | region of the cultivated land 200 from diagonally upward so that the at least 2 channel | path 220 may be included. For this reason, at least two passages 220 are drawn in the obtained original image.

  FIG. 2 is a diagram illustrating an example of a functional configuration of the work machine 100 according to the embodiment of the present invention. In FIG. 2, the same components as those shown in FIG.

  As illustrated in FIG. 2, the work machine 100 includes a photographing unit 110, an information input unit 120, a control unit 130, an image processing unit 140, a traveling mechanism 150, a crop collection mechanism 160, and a display unit 170. ing.

  Under the control of the control unit 130, the imaging unit 110 includes the culm 210 and the passage 220 in a state where the imaging direction is set in the traveling direction 101 of the work machine 100 as described above with reference to FIG. 1. An area of the ground 200 is photographed from obliquely above to obtain an original image of the area.

  The information input unit 120 inputs various types of information to the control unit 130. For example, the information input unit 120 includes switches and buttons for an operator to input an operation, and further includes a communication interface for communicating with the external device when the work machine 100 communicates with the external device. It is configured.

  The control unit 130 comprehensively controls the operation of the work machine 100 according to the information input from the information input unit 120.

  The image processing unit 140 performs image processing on the original image obtained by the photographing unit 110 based on the control of the control unit 130. As shown in FIG. 2, the image processing unit 140 includes an original image acquisition unit 141, a binarization processing unit 142, a planar projection conversion processing unit 143, an image rotation processing unit 144, an existence probability distribution acquisition unit 145, and an optimal rotation image. An extraction unit 146, an angle calculation unit 147, and a positional deviation amount calculation unit 148 are included.

  FIG. 3 is a diagram illustrating an example of various images obtained by the image processing unit 140 of FIG.

  The original image acquisition unit 141 performs processing for acquiring the original image obtained by the photographing unit 110. Here, it is assumed that the original image acquisition unit 141 acquires the original image shown in FIG. In the original image shown in FIG. 3A, a plurality of ridges 210 and a plurality of passages 220 are drawn.

  The binarization processing unit 142 binarizes the original image acquired by the original image acquisition unit 141 using the binarization threshold value, and generates a binary image. Here, in the present embodiment, when the binarization processing unit 142 performs binarization processing of the original image, the pixel value of pixels that are less than the binarization threshold among the pixels of the original image is set to 0. It is also possible to apply a mode for performing the processing as described above, but it is also possible to apply a mode described below with reference to FIG.

FIG. 4 is a diagram for explaining an example of morphological filtering performed by the binarization processing unit 142 of FIG. 2 according to the embodiment of the present invention.
Specifically, as an aspect applicable to the embodiment of the present invention, the binarization processing unit 142 expands an area less than the binarization threshold in the vertical direction 302 of the original image shown in FIG. In the horizontal direction 301 of the original image shown in FIG. 3A, morphological filtering is performed to shrink the region below the binarization threshold. This aspect will be described with reference to FIGS. 4 (a) and 4 (b).

  First, the process of the original image 302 in the vertical direction shown in FIG. 3A will be described with reference to FIG. FIG. 4A shows a target pixel 401 that is a target for determining a pixel value in a binary image, and a neighboring pixel 402 that is located near the target pixel 401 in the vertical direction 302 of the original image. Yes. Specifically, FIG. 4A shows a neighboring pixel 402 that is one pixel above and below the target pixel 401. In this case, the binarization processing unit 142, for example, in FIG. 4A, when at least one of the neighboring pixels 402 is a pixel less than the binarization threshold (a pixel having a pixel value of 0). Performs a process of setting the target pixel 401 as a pixel less than the binarization threshold (a pixel having a pixel value of 0) and expanding an area less than the binarization threshold. Although an example in which one pixel above and below the target pixel 401 is set as the neighboring pixel 402 has been described here, the present invention is not limited to this form. For example, the binarization processing unit 142 sets the upper and lower n pixels (n is a natural number of 2 or more) of the target pixel 401 as the neighboring pixels 402, and at least N (N is 1 ≦ 1) among the 2n neighboring pixels 402. When N ≦ 2n−1) or more pixels are pixels that are less than the binarization threshold (pixels having a pixel value of 0), the target pixel 401 is a pixel that is less than the binarization threshold (pixels having a pixel value of 0). ), And a process of expanding a region less than the binarization threshold is also applicable to the present invention.

  Next, the processing of the original image 301 in the horizontal direction 301 illustrated in FIG. 3A will be described with reference to FIG. FIG. 4B shows a target pixel 401 that is a target for determining a pixel value in a binary image, and a neighboring pixel 402 that is located near the target pixel 401 in the horizontal direction 301 of the original image. Yes. Specifically, FIG. 4B shows a neighboring pixel 402 that is one pixel left and right with respect to the target pixel 401. In this case, the binarization processing unit 142, for example, in FIG. 4B, when at least one of the neighboring pixels 402 is a pixel that is equal to or higher than the binarization threshold (pixel value is 1). Performs a process of setting the target pixel 401 as a pixel equal to or higher than the binarization threshold (a pixel having a pixel value of 1) and contracting an area less than the binarization threshold. Here, an example in which one pixel on the left and right of the target pixel 401 is set as the neighboring pixel 402 has been described, but the present invention is not limited to this form. For example, the binarization processing unit 142 sets the left and right m pixels (m is a natural number of 2 or more) of the target pixel 401 as the neighboring pixels 402, and among the 2m neighboring pixels 402, at least M (M is 1 ≦ 1). When M ≦ 2n−1) or more pixels are pixels having a binarization threshold value or more (pixels having a pixel value of 1), the target pixel 401 is a pixel having a binarization threshold value or more (a pixel having a pixel value of 1). ), And a process for contracting a region less than the binarization threshold is also applicable to the present invention.

  Further, in the present embodiment, for example, the binarization processing unit 142 performs filtering using a smoothing filter such as a Gaussian filter on the original image before performing binarization processing of the original image. By this filtering, the noise component of the original image can be reduced.

  FIG. 3B is a diagram illustrating a binary image generated by the binarization processing unit 142 by binarizing the original image illustrated in FIG. In the binary image shown in FIG. 3B, pixels that are less than the binarization threshold (pixels having a pixel value of 0) are shown in white, and pixels that are greater than or equal to the binarization threshold (pixels that have a pixel value of 1) are black. Is shown. In FIG. 3B, the part shown in white is considered to correspond to the passage 220. When the binary image shown in FIG. 3B is generated, the vertical direction 302 of the original image is drawn substantially along the vertical direction 302 by performing the morphological filtering described above with reference to FIG. The passage 220 can be emphasized more, and noise such as shadows of tea trees other than the passage 220 can be removed in the lateral direction 301 of the original image. Extraction accuracy can be improved.

Here, it returns to description of FIG. 2 again.
The plane projection conversion processing unit 143 performs plane projection conversion processing on the binary image generated by the binarization processing unit 142 to generate a plane projection conversion binary image. Specifically, the planar projection conversion processing unit 143 performs planar projection conversion processing on the binary image generated by the binarization processing unit 142 from the viewpoint of viewing the area of the cultivated land 200 photographed as the original image from above. Then, a planar projective transformation binary image is generated. FIG. 3C is a diagram illustrating a planar projective conversion binary image generated by performing the planar projective conversion process on the binary image illustrated in FIG. 3B in the planar projective conversion processing unit 143.

  The image rotation processing unit 144 performs image rotation processing for rotating the planar projection conversion binary image generated by the plane projection conversion processing unit 143 at a plurality of rotation angles, and according to each rotation angle at the plurality of rotation angles. A plurality of rotated images are generated. FIG. 3D is a diagram showing a plurality of rotated images generated by image rotation processing of the planar projective conversion binary image shown in FIG. 3C in the image rotation processing unit 144. In FIG. 3D, as an example, a rotation image at each rotation angle from 0 degree to +90 degrees (in this embodiment, the clockwise direction is a positive rotation angle) is shown.

  The existence probability distribution acquisition unit 145 binarizes the rotation images in the plurality of rotation images generated by the image rotation processing unit 144 among the pixels in the vertical direction orthogonal to the horizontal direction at each position in the horizontal direction. The number of pixels less than the threshold is calculated, and the existence probability distribution of the passage 220 based on the number of pixels less than the binarization threshold is acquired.

  FIG. 5 is a diagram illustrating an example of the existence probability distribution of the passage 220 in a certain rotation image acquired by the existence probability distribution acquisition unit 145 of FIG. 2 according to the embodiment of this invention. In FIG. 5, each horizontal position on the horizontal axis indicates each position in the horizontal direction 303 shown in FIG. In FIG. 5, the number of pixels on the vertical axis is the number of pixels in the vertical direction 304 shown in FIG. 3D orthogonal to the horizontal direction 303 at each position in the horizontal direction 303 shown in FIG. This indicates the number of pixels that are less than the binarization threshold (pixels indicated by white in FIG. 3D). Then, the existence probability distribution acquisition unit 145 acquires the existence probability distribution of the passage 220 as illustrated in FIG. 5 for each rotation image generated by the image rotation processing unit 144.

Here, it returns to description of FIG. 2 again.
The optimum rotated image extraction unit 146 optimizes the rotated image having the largest maximum value in the existence probability distribution of the passage 220 acquired by the existence probability distribution acquisition unit 145 among the plurality of rotation images generated by the image rotation processing unit 144. Extract as a rotated image. Specifically, the optimal rotation image extraction unit 146 optimally rotates a rotation image having the largest maximum value 501 in the existence probability distribution of the passage 220 illustrated in FIG. 5 among the plurality of rotation images generated by the image rotation processing unit 144. Extract as an image.

  The angle calculation unit 147 calculates an angle in the traveling direction 101 of the work machine 100 with respect to the rod 210 based on the rotation angle in the optimal rotation image extracted by the optimal rotation image extraction unit 146.

  In this embodiment, the positional deviation amount calculation unit 148 is arranged in the horizontal direction in the optimal rotation image extracted by the optimal rotation image extraction unit 146 because the photographing unit 110 is arranged at the center position in the orthogonal direction 102 of the work machine 100. Based on the center position in the direction 303 and the position where the two passages 220 exist in the optimum rotation image, the amount of positional deviation from the center position of the rod 210 in the work machine 100 is calculated. In the present invention, as described above, the imaging unit 110 does not necessarily have to be arranged at the center position in the orthogonal direction 102 of the work machine 100. For example, the traveling direction 101 of the work machine 100 can be taken. Since the configuration arranged at the upper part of the front portion of the work machine 100 is also applicable, in this configuration, the positional deviation amount calculation unit 148 is configured to perform the optimal rotation extracted by the optimal rotation image extraction unit 146. Based on the center position of the horizontal direction 303 in the image, the position where the two passages 220 exist in the optimum rotation image, and the position of the imaging unit 110 with respect to the work machine 100, A form for calculating the positional deviation amount is adopted.

  FIG. 6 is a diagram for explaining the positional deviation amount calculation processing by the positional deviation amount calculation unit 148 in FIG. 2 according to the embodiment of the present invention. In FIG. 6, the existence probability distribution 602 of the passage 220 acquired by the existence probability distribution acquisition unit 145 on the optimum rotation image 600 together with the optimum rotation image 600 extracted by the optimum rotation image extraction unit 146 is a hatched area. It is shown. FIG. 6 also shows a center position 601 in the horizontal direction (303) in the optimally rotated image 600 and positions 603 and 604 where two paths (220) exist in the optimally rotated image 600.

  Here, as shown in FIG. 6, it is assumed that the number of pixels between the position 603 where the passage 220 exists and the position 604 where the passage 220 exists in the optimum rotation image 600 is 290 pixels (290 pixels). Further, as shown in FIG. 6, in the optimally rotated image 600, the number of pixels between the position 603 where the passage 220 exists and the center position 601 in the horizontal direction is 105 pixels (105 pixels). It is assumed that the number of pixels between the existing position 604 and the horizontal center position 601 is 185 pixels (185 pixels). Then, since it is considered that the ridge 210 exists in a region between the position 603 where the passage 220 exists and the position 604 where the passage 220 exists, as shown in FIG. 6, the horizontal center position 601 in the optimum rotation image 600. Is shifted to the left by 40 pixels (40 pixels) with respect to the center position in the horizontal direction of the region where the ridge 210 exists. At this time, in the present embodiment, as described above, since the photographing unit 110 is disposed at the center position in the orthogonal direction 102 of the work implement 100, the center position of the work implement 100 in the orthogonal direction 102 is optimal rotation. This can be regarded as the same as the horizontal center position 601 in the image 600. For example, if it is assumed that 180/290 (cm) per pixel in the cultivated land 200, the positional deviation amount from the center position of the ridge 210 in the work machine 100 is calculated as 40 × 180 / 290≈25 cm. be able to. At this time, if the rightward misalignment is the + direction, the amount of misalignment from the center position of the collar 210 in the work machine 100 is calculated to be −25 cm.

Here, it returns to description of FIG. 2 again.
The traveling mechanism 150 is a mechanism for causing the work machine 100 to travel along the ridge 210 of the cultivated land 200 based on the control of the control unit 130. The traveling mechanism 150 includes a caterpillar or a tire. For example, the control unit 130 may determine the angle in the traveling direction 101 of the work machine 100 with respect to the heel 210 calculated by the position deviation amount calculation unit 148 and the center position of the heel 210 in the work machine 100 calculated by the position deviation amount calculation unit 148. The travel by the travel mechanism 150 is controlled so that the positional deviation amount from the travel distance becomes smaller.

  The crop harvesting mechanism 160 is a mechanism for harvesting crops planted in the straw 210 of the cultivated land 200 with the work machine 100 based on the control of the control unit 130.

  The display unit 170 displays the angle in the traveling direction 101 of the work machine 100 with respect to the heel 210 calculated by the positional deviation amount calculation unit 148 and the center position of the heel 210 in the work machine 100 calculated by the positional deviation amount calculation unit 148. Displays information on the amount of misalignment. At this time, the display unit 170 may display the original image acquired by the original image acquisition unit 141 together with the information. Then, the operator can grasp the current traveling state of the work machine 100 with respect to the eaves 210 by looking at the information displayed on the display unit 170.

  FIG. 7 is a diagram illustrating an example of a screen display by the display unit 170 of FIG. 2 according to the embodiment of the present invention. In FIG. 7, the original image acquired by the original image acquisition unit 141 is displayed in the image display area 710 of the display unit 170. In FIG. 7, traveling state information 721, angle information 722, and positional deviation amount information 723 are displayed in the information display area 720 of the display unit 170. Specifically, the traveling state information 721 displays information indicating that the vehicle is traveling. The angle information 722 displays information indicating that the angle of the work implement 100 with respect to the rod 210 in the traveling direction 101 is −4 degrees (4 degrees counterclockwise). Further, the positional deviation amount information 723 displays information indicating that the positional deviation amount from the center position of the rod 210 in the work machine 100 is −25 cm (the positional deviation is 25 cm in the left direction).

  Next, a processing procedure in the method for controlling the work machine 100 according to the embodiment of the present invention will be described. FIG. 8 is a flowchart illustrating an example of a processing procedure in the method for controlling the work machine 100 according to the embodiment of the present invention. In the description of the flowchart shown in FIG. 8, description will be made using each functional configuration of the work machine 100 shown in FIG. 2.

  In addition, at the time of starting the process of the flowchart shown in FIG. 8, the traveling of the working machine 100 by the traveling mechanism 150 and the harvesting of the crops planted in the basket 210 by the crop collecting mechanism 160 have already been performed. And

  First, in step S <b> 101, the photographing unit 110 is configured with the jar 210 in a state where the photographing direction is set to the traveling direction 101 of the work machine 100 as described above with reference to FIG. 1 based on the control of the control unit 130. An area of the cultivated land 200 including the passage 220 is photographed from obliquely above to obtain an original image of the area. Here, for example, the original image shown in FIG. Thereafter, the original image acquisition unit 141 performs processing for acquiring the original image obtained by the photographing unit 110.

  Subsequently, in step S102, the binarization processing unit 142 binarizes the original image acquired in step S101 using the binarization threshold value, and generates a binary image. At the time of this binarization processing, in the present embodiment, it is preferable to perform the morphological filtering described with reference to FIG. Here, for example, the binary image shown in FIG. 3B is generated. In the present embodiment, for example, the binarization processing unit 142 preferably performs filtering using a smoothing filter such as a Gaussian filter on the original image before performing binarization processing on the original image. .

  Subsequently, in step S103, the planar projection conversion processing unit 143 performs planar projection conversion processing on the binary image generated in step S102 to generate a planar projection conversion binary image. Specifically, the planar projection conversion processing unit 143 performs a planar projection conversion process on the binary image generated in step S102 from the viewpoint of viewing the area of the cultivated land 200 photographed as the original image from above. Then, a planar projective transformation binary image is generated. Here, for example, a planar projective transformation binary image shown in FIG.

  Subsequently, in step S104, the image rotation processing unit 144 performs image rotation processing for rotating the planar projection conversion binary image generated in step S103 at a plurality of rotation angles, and each rotation angle at the plurality of rotation angles. A plurality of rotated images corresponding to the above are generated. Here, for example, a plurality of rotated images as shown in FIG.

  Subsequently, in step S105, the existence probability distribution acquisition unit 145 determines, for each rotated image in the plurality of rotated images generated in step S104, out of the pixels in the vertical direction orthogonal to the horizontal direction at each position in the horizontal direction. The number of pixels less than the binarization threshold is calculated, and the existence probability distribution of the passage 220 based on the number of pixels less than the binarization threshold is obtained. Specifically, for example, the existence probability distribution acquisition unit 145 acquires the existence probability distribution of the passage 220 described with reference to FIG. 5 for each rotation image generated in step S104.

  Subsequently, in step S106, the optimum rotation image extraction unit 146 optimizes the rotation image having the largest maximum value in the existence probability distribution of the passage 220 acquired in step S105 among the plurality of rotation images generated in step S104. Extract as a rotated image. Specifically, the optimum rotation image extraction unit 146 extracts, as the optimum rotation image, a rotation image having the largest maximum value 501 in the existence probability distribution of the passage 220 illustrated in FIG. 5 among the plurality of rotation images generated in step S104. To do.

  Subsequently, in step S107, the angle calculation unit 147 calculates an angle in the traveling direction 101 of the work machine 100 with respect to the rod 210 based on the rotation angle in the optimum rotation image extracted in step S106.

  Subsequently, in step S108, the positional deviation amount calculation unit 148 has a center position (601 in FIG. 6) in the horizontal direction 303 in the optimally rotated image extracted in step S106 and two paths 220 in the optimally rotated image. Based on the positions to be moved (603 and 604 in FIG. 6), a positional deviation amount from the center position of the rod 210 in the work machine 100 is calculated. A detailed processing example of step S108 is as described with reference to FIG.

  Subsequently, in step S109, the display unit 170, based on the control of the control unit 130, the angle in the traveling direction 101 of the work implement 100 with respect to the rod 210 calculated in step S107, and the work implement 100 calculated in step S108. Information on the amount of positional deviation from the center position of the heel 210 is displayed. At this time, the display unit 170 may display the original image acquired in step S101 together with the information. This display example is as described with reference to FIG. Then, the operator can grasp the current traveling state of the work machine 100 with respect to the eaves 210 by looking at the information displayed on the display unit 170.

  Subsequently, in step S110, the control unit 130 determines the angle in the traveling direction 101 of the work implement 100 with respect to the rod 210 calculated in step S107 and the position from the center position of the rod 210 in the work implement 100 calculated in step S108. Based on the amount of deviation, traveling control (automatic traveling control) by the traveling mechanism 150 is performed. Specifically, the control unit 130 determines the angle in the traveling direction 101 of the work implement 100 with respect to the saddle 210 calculated in step S107 and the positional deviation amount from the center position of the saddle 210 in the work implement 100 calculated in step S108. The travel control by the travel mechanism 150 is performed so that both are small.

  In the processing example of the flowchart shown in FIG. 8, the angle in the traveling direction 101 of the work implement 100 with respect to the rod 210 calculated in step S107 and the position from the center position of the rod 210 in the work implement 100 calculated in step S108. The travel control by the travel mechanism 150 is performed so that both the deviation amounts are small, but the present invention (the present embodiment) is not limited to this mode. For example, the present invention (this embodiment) includes a mode in which travel control is performed by the travel mechanism 150 so that only the angle in the traveling direction 101 of the work implement 100 with respect to the rod 210 calculated in step S107 is reduced. Furthermore, for example, the present invention (the present embodiment) also includes an aspect in which the traveling control by the traveling mechanism 150 is performed so that only the displacement amount from the center position of the rod 210 in the work machine 100 calculated in step S108 is reduced. It is. That is, in the present invention (this embodiment), the angle in the traveling direction 101 of the work implement 100 with respect to the rod 210 calculated in step S107 and the position from the center position of the rod 210 in the work implement 100 calculated in step S108. A mode in which the traveling control by the traveling mechanism 150 is performed so that either one of the deviation amounts is small is also included.

  With the above traveling control, the traveling direction 101 of the work machine 100 can be made to follow the row of rows with high accuracy, and the crop collecting mechanism 160 can efficiently collect the crop. Note that one of the angle in the traveling direction 101 of the work implement 100 with respect to the rod 210 calculated in step S107 and the amount of positional deviation from the center position of the rod 210 calculated in step S108 are reduced. As described above, the angle in the traveling direction 101 of the work implement 100 with respect to the rod 210 calculated in step S107 and the center position of the rod 210 in the work implement 100 calculated in step S108, as compared with the mode in which the traveling control by the traveling mechanism 150 is performed. In the mode in which the traveling control by the traveling mechanism 150 is performed so that both the positional deviation amount from the head and the amount of positional deviation from each other become smaller, the traveling direction 101 of the work machine 100 can follow the row of trains with higher accuracy. The crop can be collected more efficiently by the collection mechanism 160.

  Subsequently, in step S <b> 111, the control unit 130 determines whether or not to end the automatic travel control of the work machine 100 based on information from the information input unit 120. As a result of this determination, when the automatic traveling control of the work machine 100 is not terminated (that is, the automatic traveling control is continued) (S111 / NO), the process returns to step S101, and the processes after step S101 are performed again.

  On the other hand, as a result of the determination in step S111, when the automatic travel control of the work machine 100 is to be ended (S111 / YES), the processing of the flowchart shown in FIG. 8 is ended.

Although not particularly mentioned in the above description, it is also preferable in the present embodiment to divide the process into a plurality of stages (for example, two stages) when extracting the optimum rotated image. Hereinafter, a mode in which the extraction of the optimum rotated image is divided into two steps will be described.
First, in the first stage, at the time of generating a plurality of rotated images in step S104, for example, rotated images from −90 degrees to +90 degrees with a rotation angle of 10 degrees are generated. In step S105, the existence probability distribution of the passage 220 in each rotation image in increments of 10 degrees is acquired, and then in step S106, the rotation image having the largest maximum value in the existence probability distribution of the passage 220 is extracted as the first rotation image. To do.
Subsequently, in the second stage, when a plurality of rotation images are generated in step S104 again, for example, a rotation image in a range of ± 5 degrees is generated with respect to the first rotation image in increments of 1 degree. . Then, in step S105, the existence probability distribution of the passage 220 in each rotation image in increments of 1 degree is acquired, and in step S106, the rotation image having the largest maximum value in the existence probability distribution of the passage 220 is extracted as the second rotation image. To do. Then, the second rotated image is set as an optimally rotated image, and the processes after step S107 are performed.

As described above, in the working machine 100 according to the embodiment of the present invention, the cultivated land including the ridge 210 and the passage 220 in the photographing unit 110 with the photographing direction set in the traveling direction 101 of the working machine 100. An area 200 is photographed obliquely from above, and an original image of the area is obtained. In the work machine 100 according to the embodiment of the present invention, the binarization processing unit 142 binarizes the original image using the binarization threshold value to generate a binary image, and the planar projection conversion processing unit In 143, the binary image is subjected to plane projection conversion processing to generate a plane projection conversion binary image. In the work machine 100 according to the embodiment of the present invention, the image rotation processing unit 144 performs image rotation processing for rotating the planar projection conversion binary image at a plurality of rotation angles, and each rotation angle at the plurality of rotation angles. A plurality of rotated images corresponding to the above are generated. In the work machine 100 according to the embodiment of the present invention, in the existence probability distribution acquisition unit 145, the vertical direction 304 orthogonal to the horizontal direction 303 at each position in the horizontal direction 303 for each rotation image in the plurality of rotation images. The number of pixels that are less than the binarization threshold is calculated, the existence probability distribution of the passage 220 based on the number of pixels that is less than the binarization threshold is acquired, and the optimum rotated image extraction unit 146 performs image rotation. Of the plurality of rotation images generated by the processing unit 144, the rotation image having the largest maximum value in the existence probability distribution of the passage 220 acquired by the existence probability distribution acquisition unit 145 is extracted as the optimum rotation image. In the work machine 100 according to the embodiment of the present invention, the angle calculation unit 147 calculates the angle in the traveling direction 101 of the work machine 100 with respect to the rod 210 based on the rotation angle in the optimum rotation image. Furthermore, in the work machine 100 according to the embodiment of the present invention, the positional deviation amount calculation unit 148 is based on the center position in the horizontal direction 303 in the optimum rotation image and the position where the two passages 220 exist in the optimum rotation image. Thus, the positional deviation amount from the center position of the collar 210 in the work machine 100 is calculated.
According to this configuration, by performing image processing on the original image obtained in the photographing unit 110, the angle in the traveling direction 101 of the work machine 100 with respect to the heel 210 and the amount of positional deviation from the center position of the heel 210 in the work machine 100. Therefore, the angle and the positional deviation amount can be calculated without touching the side surface of the basket 210 on which the crop is planted. As a result, automatic traveling control can be performed without destroying the pod on which the crop is planted.

(Other embodiments)
The present invention supplies a program that realizes one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and one or more processors in a computer of the system or apparatus read and execute the program This process can be realized. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.
This program and a computer-readable storage medium storing the program are included in the present invention.

  Note that the above-described embodiments of the present invention are merely examples of implementation in practicing the present invention, and the technical scope of the present invention should not be construed as being limited thereto. It is. That is, the present invention can be implemented in various forms without departing from the technical idea or the main features thereof.

100: work machine, 110: photographing unit, 120: information input unit, 130: control unit, 140: image processing unit, 141: original image acquisition unit, 142: binarization processing unit, 143: plane projection conversion processing unit, 144: Image rotation processing unit, 145: Existence probability distribution acquisition unit, 146: Optimal rotation image extraction unit, 147: Angle calculation unit, 148: Position shift amount calculation unit, 150: Travel mechanism, 160: Crop harvesting mechanism, 170: Display unit, 200: Cultivated land

Claims (8)

A work machine that travels along the cultivated land where the ridges in which the crops are planted are formed in rows,
In a state where the shooting direction is set in the traveling direction of the work implement, the area of the cultivated land including the passage between the adjacent ridges and the ridge is photographed from obliquely above to obtain an original image of the area. Photographing means;
Binarization processing means for generating a binary image by binarizing the original image using a binarization threshold;
Plane projection conversion processing means for generating a plane projection conversion binary image by performing plane projection conversion processing on the binary image;
Image rotation processing means for performing image rotation processing for rotating the planar projection conversion binary image at a plurality of rotation angles, and generating a plurality of rotation images corresponding to the rotation angles at the plurality of rotation angles;
For each rotated image in the plurality of rotated images, the number of pixels less than the binarization threshold among the pixels in the vertical direction orthogonal to the horizontal direction at each position in the horizontal direction is calculated, and the binarization threshold Presence probability distribution acquisition means for acquiring the existence probability distribution of the passage based on the number of pixels less than,
Among the plurality of rotation images, an optimum rotation image extraction means for extracting a rotation image having the largest maximum value in the existence probability distribution of the passage as an optimum rotation image;
A working machine comprising: an angle calculating unit that calculates an angle in a traveling direction of the working machine with respect to the heel based on the rotation angle in the optimum rotation image. The photographing means is arranged on the upper part of the front part of the working machine capable of photographing the traveling direction of the working machine.
In the original image, at least two of the passages are drawn,
Based on the center position in the horizontal direction in the optimum rotation image, the position where the two passages exist in the optimum rotation image, and the position of the photographing means with respect to the work machine, the center of the heel in the work machine The work machine according to claim 1, further comprising position shift amount calculation means for calculating a position shift amount from the position.   The binarization processing means expands an area less than the binarization threshold in the vertical direction of the original image and contracts an area less than the binarization threshold in the horizontal direction of the original image. The work machine according to claim 1, wherein the binary image is generated by performing filtering. A work machine that travels along the cultivated land where the ridges in which the crops are planted are formed in rows,
At least two passages between adjacent scissors are arranged in the upper part of the front part of the work machine that can photograph the travel direction of the work machine, and the shooting direction is set in the travel direction of the work machine. A photographing means for photographing an area of the cultivated land including one and the ridge from obliquely above, and obtaining an original image of the area;
Binarization processing means for generating a binary image by binarizing the original image using a binarization threshold;
Plane projection conversion processing means for generating a plane projection conversion binary image by performing plane projection conversion processing on the binary image;
Image rotation processing means for performing image rotation processing for rotating the planar projection conversion binary image at a plurality of rotation angles, and generating a plurality of rotation images corresponding to the rotation angles at the plurality of rotation angles;
For each rotated image in the plurality of rotated images, the number of pixels less than the binarization threshold among the pixels in the vertical direction orthogonal to the horizontal direction at each position in the horizontal direction is calculated, and the binarization threshold Presence probability distribution acquisition means for acquiring the existence probability distribution of the passage based on the number of pixels less than,
Among the plurality of rotation images, an optimum rotation image extraction means for extracting a rotation image having the largest maximum value in the existence probability distribution of the passage as an optimum rotation image;
Based on the center position in the horizontal direction in the optimum rotation image, the position where the two passages exist in the optimum rotation image, and the position of the photographing means with respect to the work machine, the center of the heel in the work machine A working machine comprising: a positional deviation amount calculating means for calculating a positional deviation amount from a position.   The work machine according to claim 4, further comprising angle calculation means for calculating an angle in a traveling direction of the work machine with respect to the rod based on the rotation angle in the optimum rotation image.   The binarization processing means expands an area less than the binarization threshold in the vertical direction of the original image and contracts an area less than the binarization threshold in the horizontal direction of the original image. 6. The work machine according to claim 4, wherein the binary image is generated by performing filtering. A method for controlling a working machine that travels along a cultivated land in which ridges in which crops are planted are formed in rows,
In a state in which the shooting direction is set in the traveling direction of the work implement, the area of the cultivated land including the passage between the adjacent ridges and the ridge is photographed from a diagonally upper side by a photographing unit, and the original of the area is captured. A shooting step to obtain an image;
A binarization processing step of binarizing the original image using a binarization threshold to generate a binary image;
Planar projective transformation processing step of generating a planar projective transformation binary image by performing planar projective transformation processing on the binary image;
An image rotation processing step of performing image rotation processing for rotating the planar projective transformation binary image at a plurality of rotation angles, and generating a plurality of rotation images corresponding to the rotation angles at the plurality of rotation angles;
For each rotated image in the plurality of rotated images, the number of pixels less than the binarization threshold among the pixels in the vertical direction orthogonal to the horizontal direction at each position in the horizontal direction is calculated, and the binarization threshold An existence probability distribution obtaining step of obtaining an existence probability distribution of the passage based on the number of pixels less than, and
An optimum rotation image extraction step of extracting a rotation image having the largest maximum value in the existence probability distribution of the passage among the plurality of rotation images as an optimum rotation image;
A work machine control method comprising: an angle calculation step of calculating an angle in a traveling direction of the work machine with respect to the rod based on the rotation angle in the optimum rotation image. A method for controlling a working machine that travels along a cultivated land in which ridges in which crops are planted are formed in rows,
Using the photographing means arranged at the upper part of the front part of the working machine capable of photographing the traveling direction of the working machine, the shooting direction is set in the traveling direction of the working machine, An imaging step of capturing an area of the cultivated land including at least two passages between and the ridge from an obliquely upper side to obtain an original image of the area;
A binarization processing step of binarizing the original image using a binarization threshold to generate a binary image;
Planar projective transformation processing step of generating a planar projective transformation binary image by performing planar projective transformation processing on the binary image;
An image rotation processing step of performing image rotation processing for rotating the planar projective transformation binary image at a plurality of rotation angles, and generating a plurality of rotation images corresponding to the rotation angles at the plurality of rotation angles;
For each rotated image in the plurality of rotated images, the number of pixels less than the binarization threshold among the pixels in the vertical direction orthogonal to the horizontal direction at each position in the horizontal direction is calculated, and the binarization threshold An existence probability distribution obtaining step of obtaining an existence probability distribution of the passage based on the number of pixels less than, and
An optimum rotation image extraction step of extracting a rotation image having the largest maximum value in the existence probability distribution of the passage among the plurality of rotation images as an optimum rotation image;
Based on the horizontal center position in the optimum rotation image, the position where the two passages exist in the optimum rotation image, and the position of the photographing means with respect to the work machine, the center of the heel in the work machine And a position deviation amount calculating step for calculating a position deviation amount from the position.