JP2020101943A - Field working vehicle - Google Patents

Abstract

To provide a field working vehicle which can automatically travel in consideration of a state of a field without using satellite positioning.SOLUTION: The field working vehicle comprises: a photographing unit 3 which photographs a field including at least an area where a vehicle body advances, and outputs a field image; a travel device which adjusts a vehicle speed and an amount of steering; an automatic travel operation unit 4 which is learned so as to receive the field image and output automatic travel information for automatic travel on the basis of a boundary between an unworked area and a worked area and a state of the field; and an automatic travel control unit 52 which controls the travel device on the basis of the automatic travel information.SELECTED DRAWING: Figure 5

Description

The present invention relates to a field work vehicle that performs work while automatically traveling in a field.
Regarding

In one of the methods for automatically traveling in the field, a preset traveling route for automatic traveling and a work vehicle position obtained by satellite positioning are used. For example, in the tractor disclosed in Patent Document 1, a work target area to be worked by automatic running is formed inside an already-worked area created by first running near the outer circumference of a field by manual running. .. A travel route for the tractor to travel in the work target area is calculated based on a route calculation algorithm. After that, the automatic traveling control unit provided in the contactor calculates the positional deviation between the target traveling route and the own vehicle position calculated from the satellite positioning data, and issues an automatic steering command to reduce the positional deviation. Is generated.

Another method for automatically traveling in the field uses a target travel route of the field work vehicle that is obtained by performing image processing on an image captured by a shooting camera mounted on the field work vehicle. For example, in the lawn mower disclosed in Patent Document 2, a boundary area between the already-cut land and the uncut land in a grassy area such as a golf course is photographed by a photographing device, and the photographed image shows the already-cut and uncut land. A boundary line between the two is calculated, and the vehicle body is automatically steered along the boundary line.

JP, 2008-116608, A JP, 2008-097526, A

In order to carry out automatic traveling using satellite positioning data as disclosed in Patent Document 1, map information of a field (field coordinate system or earth coordinate system) is prepared, and using this map information, A travel route that covers the field is calculated. The work vehicle is automatically driven so that the position of the vehicle does not deviate from this travel route. The own vehicle position is calculated by using satellite positioning. However, in the own vehicle position calculation by satellite positioning, if satellite waves are blocked by forests or houses, there is a problem that positioning is impossible or positioning accuracy deteriorates.

Automatic traveling using the boundary line between the already-cut land (worked land) and the uncut land (unworked land) detected by image processing as the target traveling route, as disclosed in Patent Document 2, It is effective as an alternative to autonomous driving using satellite positioning. However, the field scene that is the traveling surface of the field work vehicle has more irregularities than a lawn or the like, and the state thereof also varies depending on the location, which leads to deterioration of steering accuracy. Furthermore, when agricultural products or agricultural product residues are present in the field, the state of the agricultural products varies depending on the location. Therefore, automatic driving depending on the state of such agricultural products may be required.

An object of the present invention is to provide a field work vehicle that can automatically travel without using satellite positioning and while considering the state of the field.

A field work vehicle according to the present invention, which automatically travels in a field, includes a shooting unit for shooting the field including at least a traveling region of a vehicle body and outputting a field image, a traveling device for adjusting a vehicle speed and a steering amount, A field image is input, and an automatic travel calculation unit learned to output automatic travel information for automatic travel based on a boundary between an unworked area and an already-worked area and a field state, and the automatic travel information. And an automatic traveling control unit that controls the traveling device based on the traveling device.

In this configuration, the target traveling direction of the vehicle body is determined by estimating the boundary between the unworked area and the already-worked area from the field image acquired by the imaging unit, and at the same time, the field state is estimated from the field image. Then, the target traveling behavior suitable for the field condition is determined, and the automatic traveling information is output from the target traveling direction and the traveling behavior. The automatic traveling calculation unit is learned to output the optimum automatic traveling information including not only the boundary between the unworked area and the already-worked area but also the field condition. Therefore, when the traveling device is controlled so as to be along the boundary between the unworked area and the already-worked area, the field condition (field condition, crop condition, etc.) is also considered. For example, if there is mud or unevenness in front of the vehicle, it is possible to reduce the steering amount and suppress slippage.

In order to avoid work leakage or overlapping work (excessive overlap), it is necessary to accurately grasp the deviation of the vehicle body from the target route and obtain the steering amount that reduces the deviation. For this reason, it is important to obtain the boundary between the unworked area and the already-worked area with a single line, that is, the boundary line, and to output a steering command for matching the steering reference point of the vehicle body with this boundary line. is there. From this, in one of the preferred embodiments of the present invention, the automatic travel information estimates a linear target travel route corresponding to the boundary line indicating the boundary, It contains information to eliminate the gap with the car.

In field work such as plowing work, rice planting work, and harvesting work, the already worked outer peripheral area is used as a turning area for changing the direction of the vehicle body and is parallel to the inner area (unworked area) of the outer peripheral area. A traveling mode in which a linear route is connected by a direction change traveling such as a U-turn traveling is adopted. In order to adopt such a traveling pattern also in the automatic traveling according to the present invention, it is necessary to perform control to shift to the direction change traveling when the vehicle body reaches the outer peripheral area where the direction of the vehicle body needs to be changed. From this, in one of the preferred embodiments of the present invention, the automatic traveling information indicates that the direction change traveling route set in the vehicle body direction changeable region which is the already-worked region is used as the target traveling route. Contains information to do.

Since there may be people, animals, and running obstacles such as wells and steel towers in the field, in order to carry out automatic driving, this running obstacle should be detected and contact with the running obstacle should be made. Need to avoid. From this, in one of the preferred embodiments of the present invention, the automatic traveling calculation unit uses, as the automatic traveling information, collision avoidance information for avoiding contact with a traveling obstacle estimated from the field image. Output.

Countermeasures differ depending on whether the moving obstacle is moving or fixed on the field. It is necessary for the work vehicle to avoid obstacles that are fixed in the field. Therefore, in one of the preferred embodiments of the present invention, the collision avoidance information includes avoidance traveling information for avoiding the traveling obstacle estimated to be a non-moving obstacle.

Obstacles that move in the field may move away from the destination of the work vehicle over time, and may disappear as traveling obstacles. It is also possible to eliminate obstacles by an alarm or the like. From this, in one of the preferred embodiments of the present invention, the collision avoidance information includes at least deceleration, stoppage, and warning for avoiding contact with the traveling obstacle estimated to be a mobile obstacle. Contains information for doing one.

When the field work is performed by manual traveling, the driver appropriately operates the vehicle speed and the engine speed in consideration of the current vehicle speed and the like to achieve optimal work traveling. Such an operation is also useful in automatic driving. From this, in one of the preferred embodiments of the present invention, a traveling state detection unit that detects a traveling state from the state of the traveling device is provided, and traveling state information indicating the traveling state is stored in the automatic traveling calculation unit. The traveling state information includes data indicating at least one of a vehicle speed, a shift position, an engine speed, and an engine load factor. The automatic traveling information includes steering, vehicle speed, gear shifting, braking, A control command for instructing change of at least one of the engine speeds is included.

In agriculture, which has been developed through many years of experience of agricultural workers and technological reform of field work vehicles, in order to promote automation of field work vehicles, it is necessary to reflect many years of experience in automatic traveling control of field work vehicles. .. For this purpose, it is preferable to introduce machine learning that causes the field work vehicle to learn the experience of the farmer at the core of the control system. As a result, the ability of a skilled farmer can be realized on the computer mounted on the field work vehicle. Therefore, the automatic running calculation unit is constructed by a convolutional neural network, and is learned by using the first learning image group and the second learning image group as learning images. Is an image group including the unworked area and the already-worked area of a field, and the second learning image group is an annotation indicating the boundary between the unworked area and the already-worked area, It is an image group to which an annotation indicating a positional deviation with respect to a traveling route for automatically traveling along the boundary is added. The annotation is a meaning given to the corresponding captured image. For example, in a captured image of a field, pointing to the boundary between the unworked area and the already-worked area Is. This machine learning includes first learning and second learning. The first learning is to find a raw photographed image acquired from a viewpoint from a field work vehicle and a boundary between an unworked area and the already worked area in the photographed image, and show the boundary area as an annotation. It is learning to estimate the boundary area from the raw captured image using the annotation image. The second learning is an annotation that shows, as annotations, a raw captured image and an operation for eliminating the positional deviation of the vehicle body from the found boundary position (deviation from the target travel line) and the required positional deviation. This is a learning for estimating the position shift boundary area of the vehicle body from the raw captured image using the image and the operation for canceling the position shift. Through this learning, the automatic traveling calculation unit can output the position of the boundary line in front of the vehicle body, the displacement of the vehicle body, and, if necessary, the operation for canceling the displacement from the acquired field image.

When the displacement of the vehicle body occurs, steering is performed to match the vehicle body with the target travel route. In order to quickly and efficiently eliminate the positional deviation due to this steering, control is performed by combining lateral positional deviation, which is the lateral deviation with respect to the travel route, and azimuth deviation, which is the vehicle body azimuth deviation with respect to the direction of the travel path. Is preferably performed. This is because the field work vehicle that travels in the field has more slip and a lower vehicle speed than the vehicle that travels on the road. From this, in one of the preferred embodiments of the present invention, the positional displacement includes a lateral positional displacement that is a displacement in the transverse direction with respect to the travel route and an azimuth that is a displacement of the vehicle body orientation with respect to the direction of the travel route. The gap is included.

Furthermore, in one of the preferred embodiments of the present invention, a third learning image and a fourth learning image are further used as the learning image, and the third learning image is the already-worked area. In the vehicle body direction changeable area, and the fourth learning image is an image group to which an annotation indicating the vehicle body direction changeable area is added. In the automatic traveling calculation unit learned by using such a learning image, a direction changing region for changing the direction of the vehicle body from the current work traveling route to the next work traveling route, and the direction changing region. It becomes possible to estimate the information for the turning travel (turning travel) in the field image.

Further, in one of preferred embodiments of the present invention, a fifth learning image and a sixth learning image are further used as the learning image, and the fifth learning image is traveled in the field. The learning image is an image group including an obstacle object, and the learning image is an image group to which an annotation indicating the obstacle object is added. In the automatic traveling calculation unit learned using such a learning image, can estimate an object that is a traveling obstacle from the field image, and the automatic traveling information necessary for avoiding contact with this obstacle can be estimated. Can be output.

It is a side view of a tractor which is an example of a field work vehicle. It is a schematic plan view of the tractor for explaining the mounting position of the photographing camera. It is explanatory drawing explaining the flow of the field work by a tractor. It is a functional block diagram of a control system. It is a functional block diagram of an automatic travel calculation unit. It is a schematic diagram which shows the image for learning used for the learning process of a 1st calculating part. It is a schematic diagram which shows the image for learning used for the learning process of a 1st calculating part. It is a schematic diagram which shows the image for learning used for the learning process of a 2nd calculating part. It is a schematic diagram which shows the image for learning used for the learning process of a 2nd calculating part. It is a flow chart which shows an example of learning processing of an automatic run computing part. It is a functional block diagram in another embodiment of an automatic run computing part.

Next, a field work vehicle for automatically traveling in a field according to the present invention will be described with reference to the drawings. In the embodiment shown in FIG. 1, the work vehicle is a tractor that automatically travels in a field. In this tractor, a driving section 20 is provided in the center of the vehicle body 1 supported by the front wheels 11 and the rear wheels 12. A working device 24, which is a rotary cultivating device, is mounted on the rear portion of the vehicle body 1 via a hydraulic lifting mechanism 23. The front wheels 11 function as steering wheels, and the traveling direction of the tractor is changed by changing the steering angle. The steering angle of the front wheels 11 is changed by the operation of the steering mechanism 13. The steering mechanism 13 includes a steering motor 14 for automatic steering. During manual traveling, steering of the front wheels 11 is performed by operating a steering wheel 22 arranged in the driving unit 20.

The tractor is provided with a photographing unit 3 that photographs a field including at least a region where the vehicle body 1 is traveling and outputs a field image. In this embodiment, as shown in FIG. 2, the photographing unit 3 includes a front camera 31, a rear camera 32, a left camera 33, and a right camera 34. The front camera 31 is arranged in the front part of the vehicle body 1 and photographs the front of the vehicle body 1. The rear camera 32 is arranged at the rear portion of the vehicle body 1 and takes an image of the rear of the vehicle body 1 including the working device 24. The left camera 33 and the right camera 34 are respectively arranged on the sides of the vehicle body 1 and take images of the left side and the right side of the vehicle body 1. By converting the viewpoints of the captured images from the front camera 31, the rear camera 32, the left camera 33, and the right camera 34 and synthesizing them, it is possible to generate a bird's-eye view image with the viewpoint above the tractor. As the field image, a front image taken by the front camera 31, a field image, or both are used. The forward captured image is used as a field image including the vehicle body traveling region when moving forward, and the backward captured image by the rear camera 32 is used as a field image including the vehicle body traveling region when traveling backward. Therefore, it is assumed that the field image described below includes a front captured image, a rear captured image, and an overhead view image.

The general procedure for field work with a tractor is shown in FIG. In this example, when the tractor enters the field (#a), peripheral work traveling for performing work along the outer circumference of the field is performed manually or automatically, and an outer peripheral area that is an already-worked area is formed on the outermost peripheral side of the field. (#B). In the surrounding work traveling, the manual traveling is preferable because the turning traveling in the corner area is complicated. Except for the corner area, since the vehicle is traveling substantially straight, automatic traveling is used. At that time, the boundary line between the farm scene and the shore or ridge, which is the outer peripheral line of the farm field, is recognized in real time by the image recognition program using machine learning, and the work boundary of the work device 24 is recognized on the recognized boundary line. The tractor is automatically steered so that When the outer peripheral area formed by the peripheral work traveling becomes large enough to change the direction of the tractor (U-turn traveling), the automatic working traveling for the unworked area is started. This automatic work traveling includes forward work traveling (#c) for linearly traveling the tractor and vehicle body direction changing traveling (#c) for raising the work device 24 in the already-worked area to make the tractor U-turn in the non-working state. d) and are included. By repeating the forward work traveling and the vehicle body direction changing traveling (#e), the operation of the entire field is completed. It should be noted that in forward work traveling, the image recognition target is the boundary line between the unworked area and the already-worked area, and in the vehicle body turning traveling, the image recognition target is the boundary line (outer circumference line) of the field and the unworked area and the already-worked area. It is a boundary line with the area. Therefore, it is preferable that different image recognition programs are used for the forward work traveling and the vehicle body turning traveling. In this case, the automatic switching between the forward work traveling image recognition program and the vehicle body turning direction traveling image recognition program is performed based on the state of the work device 24 (for example, the descending state and the ascending state, the driving state and the non-driving state). be able to. Note that, depending on the type of field, the shape of the outer peripheral area is assumed in advance, and the outer peripheral area remains in the unworked state, and the unworked area inside the outer peripheral area is preceded by #c, #d, and #e. And then #b.

FIG. 4 shows a control system built in this tractor. The control system of this embodiment includes a control unit 5 including an in-vehicle ECU group and an input/output signal processing unit 50. The control unit 5 is provided with an automatic travel calculation unit 4 constructed by applying machine learning in addition to a functional unit that executes normal work travel. In the input/output signal processing unit 50, a functional unit that processes input/output signals handled during work traveling of the tractor is built. The control unit 5 and the input/output signal processing unit 50 are connected by an in-vehicle LAN.

To the input/output signal processing unit 50, a traveling equipment group 91, a work equipment group 92, a notification device group 93, an automatic/manual switching operation tool 94, a traveling state detection sensor group 81, and a work state detection sensor group 82 are connected. .. The traveling device group 91 includes the steering motor 14 (see FIG. 1) as a steering adjusting device, a gear shift adjuster as a vehicle speed adjusting device, an engine speed adjuster, and the like. The work device group 92 includes control devices for driving the work device 24 and the elevating mechanism 23 (see FIG. 1). The notification device group 93 includes a display, a speaker, a lamp, a buzzer, and the like. The automatic/manual switching operation tool 94 is a switch that selects one of an automatic traveling mode in which the vehicle travels by automatic steering and a manual steering mode in which the vehicle travels by manual steering.

The traveling state detection sensor group 81 includes sensors that detect traveling states such as the steering amount, the engine speed, and the shift position. The work state detection sensor group 82 includes sensors that detect the posture and drive state of the work device 24.

In addition, the image pickup unit 3 is connected to the input/output signal processing unit 50. The image capturing unit 3 includes an image processing unit 30 that performs image processing on the images captured by the four cameras 31 to 34 described above to generate a field image suitable for the processing in the control unit 5. The field image output from the imaging unit 3 is input to the control unit 5 via the input/output signal processing unit 50.

The control unit 5 includes a manual traveling control unit 51, an automatic traveling control unit 52, a work control unit 53, a traveling state detection unit 54, a working state detection unit 55, and an automatic traveling calculation unit 4. In the manual travel mode, the manual travel control unit 51 controls the traveling device group 91 based on the operation by the driver. In the automatic travel mode, the automatic travel control unit 52 controls the traveling device group 91 based on the automatic travel information output from the automatic travel calculation unit 4. The work control unit 53 manually or automatically controls the work equipment group 92. The traveling state detection unit 54 detects the traveling state based on the detection signal from the traveling state detection sensor group 81. The work state detection unit 55 detects the work state based on the detection signal from the work state detection sensor group 82.

In the learned state, the automatic travel calculation unit 4 inputs the field image, estimates the boundary between the unworked area and the already-worked area, and the field status, and outputs automatic travel information for automatic travel. .. In this embodiment, the automatic travel calculation unit 4 is composed of a first calculation unit 41, a second calculation unit 42, and a selection unit 40, as shown in FIG. The first calculation unit 41 includes a first front stage calculation unit 41a and a first rear stage calculation unit 41b. The second calculation unit 42 includes a second front stage calculation unit 42a and a second rear stage calculation unit 42b.

The first front-stage calculation unit 41a estimates the boundary line indicating the boundary between the unworked area and the already-worked area and the field state from the field image. The first post-stage calculation unit 41b calculates a linear target travel route (forward work travel route) corresponding to the estimated boundary line in real time. At this time, the work width of the work device 24, the work overtap amount, the distance from the vehicle body reference point to the work end of the work device 24, and the like are necessary, and thus the work device specification data is set in advance. Further, the first front-stage computing unit 41a calculates the positional deviation of the steering reference point of the vehicle body 1 with respect to the target traveling path (forward working traveling path), and the first automatic travel information which is information for eliminating this positional deviation. To generate. The positional deviation includes a lateral positional deviation that is a positional deviation in the transverse direction with respect to the target travel route and an azimuth deviation that is an angular deviation between the azimuth of the target travel path and the vehicle body orientation. The first automatic travel information includes the steering amount for eliminating the displacement. When the steering amount is generated, the field condition such as the inclination of the field or the unevenness of the field scene is considered. The first front stage calculation unit 41a and the first rear stage calculation unit 41b can be integrated.

The second front-stage calculation unit 42a estimates a vehicle body direction changeable area, which is an already-worked area, from the field image. The second rear-stage calculation unit 42b sets a direction change travel route set in the vehicle body direction changeable area, sets a difference from an actual direction change trajectory with respect to the set direction change travel route as a turning deviation, and this turning deviation. The second automatic travel information, which is information (steering amount) for eliminating the above, is generated. The field condition such as the inclination of the field or the unevenness of the field is also taken into consideration when the second automatic travel information is generated. The second front stage calculation unit 42a and the second rear stage calculation unit 42b can be integrated.

The selection unit 40 compares the reliability of the first automatic travel information with the reliability of the second automatic travel information, selects the information with the higher appropriateness as the travel control that must be performed at the present time, and automatically travels. It is output as information and given to the automatic travel control unit 52. Alternatively, the selection unit 40 may select either the first automatic travel information or the second automatic travel information based on the work state detected by the work state detection unit 55. For example, if the state of the work device 24 is the lowered state, the driven state, or both, it is regarded as the forward work traveling, and the first automatic traveling information is selected. If the state of the work device 24 is the raised state and/or the non-driven state, it is regarded as the direction change traveling, and the second automatic traveling information is selected. As yet another method, the use of either the first front stage calculation unit 41a or the second front stage calculation unit 42a may be determined based on the work state detected by the work state detection unit 55.

In addition, in the automatic travel calculation unit 4 shown in FIG. 5, as an additional function, the travel state information detected by the travel state detection unit 54 is input to the first calculation unit 41 and the second calculation unit 42. There is. The traveling state information includes a vehicle speed, a shift position, an engine speed, an engine load factor, and the like. When this function is added, the first rear-stage calculation unit 41b and the second rear-stage calculation unit 42b add the vehicle speed control command to the first automatic traveling information and the second automatic traveling information in addition to the information for eliminating the positional deviation. It is also possible to include at least one of a shift control command, a braking control command, and an engine control command. This is because adjusting not only the steering amount but also other traveling states (vehicle speed, etc.) may be effective in eliminating the positional deviation.

The 1st calculating part 41 is a machine learning control unit which outputs the 1st automatic run information from a field image, and the 2nd calculating part 42 is a machine learning control unit which outputs the 2nd automatic run information from a field image. In order for the machine learning control unit to estimate the situation of field work from the field image and output appropriate automatic travel information, learning processing is required. For this learning processing, in FIG. 5, the learning processing unit 6 is connected to the automatic traveling calculation unit 4. The learning processing unit 6 may be built in the automatic traveling calculation unit 4. Further, a learning system may be adopted in which the learning processing unit 6 and the automatic traveling calculation unit 4 built in a computer system other than the control system of the tractor exchange data to execute the learning processing. In this embodiment, the first calculation unit 41 and the second calculation unit 42 are constructed by a convolutional neural network, and the learning processing unit 6 performs learning processing suitable for the convolutional neural network.

Since the configuration of the learning processing unit 6 is known, a detailed description thereof will be omitted here, but it has an error calculation function and a weight adjustment function. The learning processing unit 6 may include a convolutional neural network similar to the first arithmetic unit 41 or the second arithmetic unit 42, or may be directly connected to the pre-learning first arithmetic unit 41 or the second arithmetic unit 42. , May be sequentially learned. In the former case, after the learning process is completed, the learned data such as the weighting coefficient is set in the first calculation unit 41 and the second calculation unit 42.

In the learning process of the first calculation unit 41, a first learning image group including a large number of first learning images and a second learning image group including a large number of second learning images are used.

The first learning image group is an image group including an unworked area and an already-worked area of the field, and in this embodiment, the bird's-eye view image generated by the image processing unit 30 of the imaging unit 3 is used. The second learning image group is an image group to which an annotation indicating a feature to be recognized in the first learning image group is added. An example of the first learning image is shown on the left side of FIGS. 6 and 7, and a second learning image group is shown on the right side of FIGS. 6 and 7. The first learning image in FIG. 6 shows a scene spreading in the unworked area in front of the tractor located in the already worked area. In the second learning image of FIG. 6, an annotation indicating a boundary between an unworked area and an already-worked area that serves as a reference of a traveling route and a boundary line obtained from the boundary, a boundary that serves as a start reference for work traveling, and Annotation indicating the boundary line obtained from the boundary, annotation indicating the position deviation when the boundary line is the target travel route, annotation indicating that the surface condition of the unworked area is moderate unevenness, required steering amount Annotation and is added. The first learning image in FIG. 7 is a bird's-eye view image of a tractor that has entered the unworked area and is running along the boundary between the working area and the already-worked area. In the second learning image, an annotation indicating a boundary that is a reference of the traveling route (forward and backward) and a boundary line obtained from the boundary, an annotation that indicates a positional deviation when the boundary is the target traveling route, and An annotation indicating that the surface state of the work area is moderately uneven and an annotation indicating the necessary steering amount are added.

In the learning process of the second calculation unit 42, the third learning image and the fourth learning image are used as learning images. The third learning image group is also an image group including an unworked area and an already-worked area in the field, but the already-worked area in front of the vehicle body 1 is an area in which the vehicle body can be turned, and the fourth learning image. Is an image group to which an annotation indicating a vehicle body direction changeable region to be recognized in the third learning image group is added. An example of the third learning image is shown on the left side of FIGS. 8 and 9, and a fourth learning image group is shown on the right side of FIGS. 8 and 9. In the third learning image of FIG. 8, a scene spreading in the already-worked area is shown in front of the tractor during work traveling. In the fourth learning image of FIG. 8, an annotation indicating that the already-worked area that spreads forward is the vehicle body direction changeable area and the surface state of the vehicle body direction changeable area is medium unevenness. An annotation and an annotation indicating the required steering amount are added. Further, in the fourth learning image, when the vehicle body direction changeable area is recognized, the turning travel route for the vehicle body direction change set in the vehicle body direction changeable area is shown. The third learning image in FIG. 9 is a bird's-eye view image near the end of turning traveling for turning the vehicle body. In the fourth learning image of FIG. 9, an annotation indicating a boundary that is a reference of the travel route and a boundary line obtained from the boundary, an annotation that indicates a positional deviation when the boundary line is the target travel route, an already-worked area, and An annotation indicating that the surface state of the non-working area is moderate unevenness and an annotation indicating the necessary steering amount are added.

FIG. 10 is a flow of a learning process for causing the first calculation unit 41 to output the first automatic traveling information indicating the steering amount by using the first learning image group and the second learning image group. It is shown.

First, the first learning image group is acquired by using a CG device or a captured image acquired by a test run of a tractor or a drone with a camera (#01). An annotation indicating a characteristic region to be noticed is added to the first learning image group to create a second learning image group (#02).

The second learning images are sequentially extracted from the second learning image group and are input to the convolutional neural network forming the first calculation unit 41 (#11). Using the second learning image, the automatic driving information estimated by the convolutional neural network is output (#21). A difference (error) between the estimated automatic travel information and the automatic travel information indicated by the annotation attached to the second learning image is calculated (#22). If the calculated error is not less than the minimum value (#23 No branch), the weighting coefficient is corrected (#24), the process returns to step #21, and the process of reducing the error is repeated. If the calculated error is less than the minimum value (Yes branch of #23), the weighting factor at that time is recorded (#25).

It is checked whether the learning process using all the prepared second learning images has been completed (#26). If the second learning image used for the learning process still remains (#26 Yes branch), the process returns to step #11 to continue the learning process. If the second learning image used for the learning process does not remain (#26 No branch), the final weighting coefficient is determined based on the recorded weighting coefficient (#31), and the weighting coefficient is convolved with the convolutional neural network. It is set in the first calculation unit 41, which is a network (#32). Similar images may be grouped from the second learning image group, and a weighting factor that minimizes the error may be assigned to each group.

The learning process of the second calculation unit 42 using the third learning image group and the fourth learning image group is also performed by a method similar to the learning process of the first calculation unit 41 described above. When the learned second operation unit 42 recognizes that the previous work area is the area in which the vehicle body direction can be changed, the second operation unit 42 sets the turning travel route for the vehicle body direction change, changes the direction, and then It functions to output the second automatic travel information indicating the steering amount for entering the target travel route.

[Another embodiment]
(1) In the above-described embodiment, the automatic traveling calculation unit 4 is learned to output the first automatic traveling information for performing work traveling in a straight line (including a gently curved line). 41, and a second calculation unit 42 learned to output the first automatic travel information for performing the direction change travel at an appropriate place. It is convenient for the field work vehicle to have a function of automatically avoiding contact with an object (running obstacle) that causes a running obstacle. FIG. 11 shows a block diagram of the automatic traveling arithmetic unit 4 to which a third arithmetic unit 43 having a function of avoiding a traveling obstacle is added. The third calculation unit 43 includes a third front stage calculation unit 43a and a third rear stage calculation unit 43b. The third front-stage calculation unit 43a estimates the presence of a traveling obstacle from the field image (front image or bird's-eye view image) sent from the imaging unit 3. The third rear-stage computing unit 43b generates third automatic travel information including avoidance travel information (steering amount, deceleration command, stop command, warning, etc.) for avoiding contact between the estimated travel obstacle and the vehicle body 1. Output. At that time, when avoiding the traveling obstacle by steering, the avoidance traveling route is set as the target traveling route. Note that the field condition such as the inclination of the field or the unevenness of the field may be taken into consideration when the third automatic travel information is generated. The third pre-stage arithmetic unit 43a and the third post-stage arithmetic unit 43b can be integrated.

The third pre-stage arithmetic unit 43a and the third post-stage arithmetic unit 43b are configured by a convolutional neural network, and thus require learning processing. The learning image includes a field image (fifth learning image group) in which a traveling obstacle exists in the field, an annotation indicating a traveling obstacle that should be recognized in the field image, and traveling for avoiding the traveling obstacle. Is an image group (sixth learning image group) to which an annotation indicating is added. It should be noted that different learning images are prepared so that mobile obstacles (humans and animals) and non-moving obstacles (steel towers, etc.) can be distinguished and recognized as traveling obstacles. The learning process is performed according to the flowchart of FIG.

(2) In the above-described embodiment, the steering amount is included in the first automatic traveling information, the second automatic traveling information, and the third automatic traveling information, but instead of this, the operation command such as the steering amount is automatically transmitted. It may be generated by another calculation unit that derives the steering amount from the traveling information. In this case, learning for estimating the steering amount from the field information is unnecessary.

(3) In the embodiment described above, the convolutional neural network is used as the machine learning for constructing the automatic traveling calculation unit 4, but other artificial neural networks may be used. Further, the pre-stage arithmetic unit and the post-stage arithmetic unit may be constructed by the same type of machine learning, or may be constructed by different types of machine learning.

(4) When the learning processing unit 6 is provided in the control system of each field work vehicle and is connected to the automatic traveling calculation unit 4, the captured image acquired by the imaging unit 3 during the manual traveling is used as the learning image. Can be used as The annotation can be given by the driver. Furthermore, annotations may be automatically added based on information such as the operation by the driver and the travel path.

(5) In the above-described embodiment, the traveling direction of the vehicle body 1 is changed by adjusting the steering angle of the front wheels 11, which are the steered wheels, based on the steering command. In a work vehicle equipped with a crawler type traveling mechanism, instead of this, the traveling direction of the vehicle body 1 is changed by adjusting the speeds of the left and right crawlers based on a steering command.

(6) In the above-described embodiment, the tractor is handled as the field work vehicle, but the same automatic traveling calculation unit 4 can be adopted for other field work vehicles such as rice transplanters and combine harvesters.

INDUSTRIAL APPLICABILITY The present invention can be applied to a field work vehicle that automatically travels in a field.

3: Imaging unit 30: Image processing unit 31: Front camera 32: Rear camera 33: Left camera 34: Right camera 4: Automatic traveling calculation unit 40: Selection unit 41: First calculation unit 41a: First front stage calculation unit 41b: First post-stage arithmetic unit 42: Second arithmetic unit 42a: Second front-stage arithmetic unit 42b: Second post-stage arithmetic unit 43: Third arithmetic unit 43a: Third front-stage arithmetic unit 43b: Third post-stage arithmetic unit 5: Control unit 50 : Input/output signal processing unit 52: Automatic traveling control unit 54: Running state detection unit 55: Work state detection unit 6: Learning processing unit

Claims (11)

A field work vehicle that automatically travels in a field,
A photographing unit for photographing the farm field including at least the traveling region of the vehicle body and outputting a farm field image,
A traveling device for adjusting the vehicle speed and the steering amount,
By inputting the field image, estimating the boundary between the unworked area and the already-worked area and the field state, and an automatic travel calculation unit learned to output automatic travel information for automatic travel,
An automatic traveling control unit that controls the traveling device based on the automatic traveling information,
Field work vehicle equipped with. The field work vehicle according to claim 1, wherein the automatic travel information includes information for eliminating a deviation between a vehicle and a linear target travel route corresponding to a boundary line indicating the boundary. The field work according to claim 1 or 2, wherein the automatic travel information includes information for performing a direction change using a direction change travel route set in a vehicle body direction changeable region that is the already-worked region as a target travel route. car. The field according to any one of claims 1 to 3, wherein the automatic traveling calculation unit outputs collision avoidance information for avoiding contact with a traveling obstacle estimated from the field image as the automatic traveling information. Work vehicle. The field work vehicle according to claim 4, wherein the collision avoidance information includes avoidance traveling information for avoiding the traveling obstacle estimated to be a non-movable obstacle. The collision avoidance information includes information for performing at least one of deceleration, stopping, and warning for avoiding contact with the traveling obstacle estimated to be a mobile obstacle. The listed field work vehicle. A traveling state detection unit for detecting the traveling state from the state of the traveling equipment is provided,
Running state information indicating the running state is input to the automatic running calculation unit, and the running state information includes data indicating at least one of a vehicle speed, a gear shift position, an engine speed, and an engine load factor,
The field work according to any one of claims 1 to 6, wherein the automatic travel information includes a control command for instructing a change of at least one of steering, vehicle speed, speed change, braking, and engine speed. car. The automatic running calculation unit is constructed by a convolutional neural network, and is learned by using a first learning image group and a second learning image group as learning images, and the first learning image group is An image group including the unworked area and the already-worked area of a field, wherein the second learning image group includes an annotation indicating the boundary between the unworked area and the already-worked area, and the boundary. The field work vehicle according to any one of claims 1 to 7, which is an image group to which an annotation indicating a positional deviation with respect to a traveling route for automatically traveling along the is added. 9. The field work vehicle according to claim 8, wherein the positional deviation includes a lateral positional deviation that is a lateral displacement with respect to the traveling path and an azimuth deviation that is a vehicle body azimuth deviation with respect to the direction of the traveling path. .. A third learning image and a fourth learning image are further used as the learning images, and the third learning image is an image group showing a region in which the vehicle body direction can be changed in the already-worked region, The field work vehicle according to claim 8 or 9, wherein the fourth learning image is an image group to which an annotation indicating the vehicle body direction changeable region is added. A fifth learning image and a sixth learning image are further used as the learning images, and the fifth learning image is an image group including an object that is a traveling obstacle in the field, The field work vehicle according to any one of claims 8 to 10, wherein the learning image is a group of images to which an annotation indicating an object that becomes the traveling obstacle is added.

Images