JP7399680B2 - Work support system - Google Patents

Description

The present invention relates to a work support system that supports work by a work vehicle such as a tractor or a riding management machine.

A tractor, which is an example of a work vehicle, is equipped with a dedicated plant sensor device that acquires growth information (information about the growth status) of crops (plants). There is a device configured to measure the growth status of crops grown in a field (for example, see Patent Document 1).

The plant sensor device described in Patent Document 1 irradiates the same irradiation area where a crop whose growth status is to be measured is cultivated with a first measurement light and a second measurement light, and detects the first measurement light from the crop whose growth status is to be measured. The reflected lights of the first measurement light and the second measurement light are acquired. Then, using the measurement light and reflected light, a normalized difference vegetation index (NDVI), which is an example of a spectral vegetation index, and a plant height value are obtained as growth information of the crop whose growth status is to be measured. .

As a result, in the technology described in Patent Document 1, it is possible to determine the amount of fertilizer applied according to the growth situation of crops based on the normalized vegetation index and plant height value acquired by the plant sensor device. . As a result, when a tractor is equipped with a fertilizer spreader, it is possible to support work by adjusting the amount of fertilizer applied to crops according to the growing conditions of the crops.

Patent No. 6526474

In the technology described in Patent Document 1, in order to enable the above-mentioned work support, a dedicated plant sensor device is installed on the tractor to obtain normalized vegetation index, plant height value, etc., which are examples of growth information. Therefore, in order to enable work support, costs will rise and the configuration will become more complicated.

In view of this situation, the main object of the present invention is to enable work support using crop growth information while suppressing rising costs and complicating the configuration.

A first characteristic configuration of the present invention is that in a work support system,
an imaging unit that is mounted on a work vehicle working in a field and captures visible light in an imaging range set around the work vehicle to obtain color image information;
Mounted on the work vehicle, acquiring measurement information including reflection intensity of near-infrared light by projecting and receiving near-infrared light to a measurement range set around the work vehicle, and based on the acquired measurement information. an obstacle sensor that detects obstacles;
a work support unit that supports work by the work vehicle based on the color image information and the measurement information;
a positioning unit that measures the position of the work vehicle and obtains position information;
a growth information acquisition unit that acquires growth information of crops grown in the field based on the color image information and the measurement information;
The present invention further includes a storage unit that stores the growth information in association with the position information.

According to this configuration, the work support unit provides work support such as displaying an image of the surroundings of the work vehicle on the display unit based on the color image information of the imaging unit to make it easier to visually recognize the situation around the work vehicle. becomes possible. Further, the work support unit provides work support such as, for example, notifying the presence of an obstacle around the work vehicle based on the detection of the obstacle by the obstacle sensor to make it easier to avoid collision with the obstacle. becomes possible. In support of work related to obstacles, since the measurement information includes the reflection intensity of near-infrared light, the obstacle sensor can detect floating particles such as dust and fog that have occurred in the vicinity with very low reflection intensity. It is possible to avoid the possibility of erroneously detecting objects as obstacles.

The growth information acquisition unit acquires crop growth information using the color image information and measurement information acquired by the work support unit to enable the above-mentioned work support. The storage unit stores the crop growth information acquired by the growth information acquisition unit in association with the position information of the positioning unit.

As a result, for example, the growth status of crops in each predetermined area in a field can be evaluated based on the crop growth information stored in association with the position information in the storage unit, and the growth status of crops in each predetermined area evaluated can be evaluated. Based on the situation, the amount of fertilizer applied to crops in each predetermined area can be calculated. Then, when performing fertilizer spreading work using a work vehicle for fertilizer spreading, it is possible to adjust the amount of fertilizer applied to crops in each predetermined area based on the calculated amount of fertilizer. It is possible to improve the variation in the growth state of crops, and it is possible to improve the quality of crops grown in the field and stabilize the yield.

In other words, the imaging unit and obstacle sensor are used to enable work support such as making it easier to see the surroundings of the work vehicle and making it easier to avoid collisions with obstacles. With this, it is possible to acquire crop growth information without having to provide a dedicated sensor for acquiring crop growth information. As a result, by acquiring crop growth information, it is possible to provide support for improving crop quality and stabilizing yields while suppressing rising costs and complicating configurations. I can do it.

The second characteristic configuration of the present invention is
The obstacle sensor is a lidar sensor that three-dimensionally measures the distance to the measurement object existing in the measurement range,
The growth information acquisition unit acquires a plant height value of the crop as the growth information based on the distance information of the lidar sensor.

According to this configuration, since the lidar sensor has high ranging accuracy, it is possible to obtain highly accurate plant height values of crops as crop growth information. Then, based on the acquired crop growth information including the plant height value of the crops, it is possible to more accurately evaluate the crop growth status in each predetermined area in the field, and based on the evaluated crop growth status in each predetermined area As a result, the appropriate amount of fertilizer applied to crops in each predetermined area can be calculated with higher accuracy.

As a result, work support for improving crop quality and stabilizing harvest amount can be more appropriately provided based on crop growth information including highly accurate crop plant height values.

The third characteristic configuration of the present invention is
The growth information acquisition unit acquires the reflectance of visible red light from the color image information, and the reflectance of near-infrared light from the measurement information, and generates a normalized vegetation index as the growth information. It's on point to get.

According to this configuration, it is possible to properly evaluate the growth status of crops in each predetermined area in the field based on the crop growth information including the typical normalized vegetation index as crop growth information, and Based on the growth status of crops in each area, it is possible to calculate the appropriate amount of fertilizer applied to crops in each predetermined area.

As a result, work support for improving crop quality and stabilizing yields can be appropriately provided based on crop growth information including normalized vegetation indices.

The fourth characteristic configuration of the present invention is
The work support unit includes an automatic travel control unit that automatically travels the work vehicle according to a target route generated according to the field based on the position information,
The growth information acquisition unit acquires the growth information for each predetermined area set by dividing the field into a plurality of areas as the work vehicle automatically travels.

According to this configuration, it is possible to acquire crop growth information for each predetermined area in a field without requiring the effort of manually driving a work vehicle in the field.

As a result, work support for improving crop quality and stabilizing yields can be provided while eliminating the need for labor to acquire crop growth information.

The fifth characteristic configuration of the present invention is
The growth information acquisition unit calculates the amount of fertilization for each predetermined area based on the growth information for each predetermined area,
The automatic travel control section automatically adjusts the amount of fertilizer applied to each of the predetermined areas based on the position information and the amount of fertilizer applied when the work vehicle automatically travels to perform the fertilization work.

According to this configuration, fertilization work according to crop growth information for each predetermined area in the field can be performed by automatically driving the work vehicle.

As a result, it is possible to provide excellent work support for improving the quality of crops and stabilizing harvest amounts, while significantly reducing the amount of labor required by the user.

Diagram showing the schematic configuration of an automatic driving system for work vehicles Plan view of the tractor showing the imaging range of each camera Side view of the tractor showing the measurement range of each obstacle sensor, etc. A plan view of the tractor showing the measurement range of each obstacle sensor, etc. Plan view showing an example of a target route for automatic driving Block diagram showing the schematic configuration of an automatic driving system for work vehicles Block diagram showing the schematic configuration of the obstacle detection system, etc. A plan view showing the positional relationship between the mounting position of each camera and the vehicle body coordinate origin and distance calculation reference point. Diagram showing the detection range and non-detection range of obstacles in the distance image of the first obstacle sensor A diagram showing the detection range and non-detection range of obstacles in the distance image of the second obstacle sensor when the work equipment is in a lowered state. A diagram showing the detection range and non-detection range of obstacles in the distance image of the second obstacle sensor when the work equipment is in the raised state. Flowchart showing the control operation of the information integration processing unit in obstacle information acquisition control Flowchart showing the control operation of the obstacle control unit in the first collision avoidance control Flowchart showing the control operation of the automatic driving control unit in the dirt handling control process A plan view showing multiple growth information acquisition areas in a field. A diagram showing an example of a color image of the front of the vehicle body generated from color image information captured by the front camera of the imaging unit. A diagram showing an example of a distance image in front of the vehicle body generated from measurement information of the first obstacle sensor A diagram showing an example of a near-infrared image of the front of the vehicle body generated from the measurement information of the first obstacle sensor Flowchart showing the control operation of the growth information acquisition unit in growth information acquisition control

EMBODIMENT OF THE INVENTION Hereinafter, an example of the form for implementing this invention is demonstrated based on drawing.

As shown in FIGS. 1 to 5, the work vehicle V illustrated in this embodiment includes a tractor 1, which is an example of a traveling vehicle body, and an example of a work device that is removably connected to the rear part of the tractor 1 via a link mechanism 2. It has a rotary tilling device 3 which is. As a result, the work vehicle V is configured to have a rotary tillage specification that allows the rotary tiller 3 to carry out tillage work. The rotary tiller 3 is connected to the rear part of the tractor 1 via a link mechanism 2 so as to be movable up and down and rollable.

Note that the traveling vehicle body may be a riding management machine or the like having a high minimum ground clearance other than the tractor 1. The working device may be a plow, a disc harrow, a cultivator, a subsoiler, a seeding device, a spraying device, a mowing device, etc. other than the rotary tilling device 3.

The tractor 1 can be automatically driven in a field A shown in FIG. 5 or the like by using an automatic driving system for work vehicles. As shown in FIGS. 1 and 6, an automatic driving system for a work vehicle includes an automatic driving unit 4 mounted on a tractor 1, and a wireless communication device configured to communicate wirelessly with the automatic driving unit 4. A mobile communication terminal 5, which is an example, is included. The mobile communication terminal 5 is equipped with a multi-touch display device (for example, a liquid crystal panel) 50 that enables various information display and input operations related to automatic driving.

Note that the mobile communication terminal 5 may be a tablet-type personal computer, a smartphone, or the like. Furthermore, wireless communication may include wireless LAN (Local Area Network) such as Wi-Fi (registered trademark), short-range wireless communication such as Bluetooth (registered trademark), and the like.

As shown in FIGS. 1 to 3 and FIG. 6, the tractor 1 includes drivable and steerable left and right front wheels 10, drivable left and right rear wheels 11, a cabin 13 forming a riding-type driving section 12, and a common rail. An electronically controlled diesel engine (hereinafter referred to as engine) 14 having a system, a hood 15 that covers the engine 14, etc., a transmission unit 16 that changes the speed of the power from the engine 14, and the like are provided. Note that the engine 14 may be an electronically controlled gasoline engine having an electronic governor.

As shown in FIG. 6 , the tractor 1 includes a fully hydraulic power steering unit 17 that steers the left and right front wheels 10 , a brake unit 18 that brakes the left and right rear wheels 11 , and a brake unit 18 that brakes the left and right rear wheels 11 . An electro-hydraulic controlled work clutch unit 19, an electro-hydraulic controlled lifting drive unit 20 that drives the rotary tilling device 3 up and down, and an electro-hydraulic controlled rolling unit 21 that enables the rotary tilling device 3 to be driven in the roll direction. , a vehicle condition detection device 22 including various sensors and switches for detecting various setting conditions and operating conditions of each part of the tractor 1, and an on-vehicle control unit 23 having various control sections. . Note that the power steering unit 17 may be an electric type having an electric motor for steering.

As shown in FIGS. 1 and 3, the driving section 12 includes a steering wheel 25 for manual steering, a seat 26 for passengers, and an operating terminal 27 that allows various information display and input operations. It is equipped. Although not shown in the drawings, the driving unit 12 is equipped with operating levers such as an accelerator lever and a gear shift lever, and operating pedals such as an accelerator pedal and a clutch pedal. As the operation terminal 27, a multi-touch liquid crystal monitor, a virtual terminal compatible with ISOBUS, or the like can be adopted.

Although not shown, the transmission unit 16 includes an electronically controlled continuously variable transmission that changes the speed of the power from the engine 14, and an electronically controlled continuously variable transmission that switches the power after shifting by the continuously variable transmission between forward and reverse. It includes a hydraulically controlled forward/reverse switching device. Continuously variable transmissions include I-HMT (Integrated Hydro-static Mechanical Transmission), which is an example of a hydro-mechanical continuously variable transmission that has higher transmission efficiency than Hydrostatic Transmission (HST). It has been adopted. The forward/reverse switching device includes a hydraulic clutch for intermittent forward power, a hydraulic clutch for intermittent reverse power, and an electromagnetic valve that controls the flow of oil to these clutches.

Note that, instead of the I-HMT, the continuously variable transmission may be an HMT (Hydraulic Mechanical Transmission), which is an example of a hydromechanical continuously variable transmission, a hydrostatic continuously variable transmission, or a belt-type continuously variable transmission. , etc. may be adopted. Furthermore, instead of the continuously variable transmission, the transmission unit 16 includes an electrohydraulic-controlled stepped transmission that has a plurality of hydraulic clutches for shifting and a plurality of electromagnetic valves that control the flow of oil to the hydraulic clutches. It may be

Although not shown, the brake unit 18 includes left and right brakes that individually brake the left and right rear wheels 11, and left and right brakes that operate in conjunction with depression of left and right brake pedals provided in the driving section 12. A foot brake system, a parking brake system that operates the left and right brakes in conjunction with the operation of a parking lever provided in the driving section 12, and a parking brake system that operates the brakes on the inside of the turn in conjunction with the steering of the left and right front wheels 10 at a set angle or more. It includes a swing brake system to operate, etc.

The vehicle condition detection device 22 is a general term for various sensors, switches, etc. provided in each part of the tractor 1. As shown in FIG. 7, the vehicle condition detection device 22 includes a vehicle speed sensor 22A that detects the vehicle speed of the tractor 1, a reverser sensor 22B that detects the operating position of a reverser lever for switching forward and backward, and a steering angle of the front wheels 10. A steering angle sensor 22C that detects the steering angle is included. Although not shown, the vehicle condition detection device 22 includes a rotation sensor that detects the output rotation speed of the engine 14, an accelerator sensor that detects the operating position of the accelerator lever, and a shift sensor that detects the operating position of the gear shift lever. Contains sensors, etc.

As shown in FIGS. 6 and 7, the on-vehicle control unit 23 includes an engine control section 23A that controls the engine 14, and a transmission unit control section 23B that controls the transmission unit 16, such as switching the vehicle speed of the tractor 1 and forward/backward movement. , a steering control section 23C that controls steering, a working device control section 23D that controls working devices such as the rotary tiller 3, a display control section 23E that controls displays and notifications on the operation terminal 27, etc., and a control related to automatic driving. an automatic driving control unit 23F that functions as a work support unit by performing the following, and a non-volatile in-vehicle storage unit 23G that stores a target route P for automatic driving (see FIG. 5) generated according to the field A, etc. etc. are included. Each of the control units 23A to 23F is constructed by an electronic control unit in which a microcontroller and the like are integrated, various control programs, and the like. Each of the control units 23A to 23F is connected to be able to communicate with each other via a CAN (Controller Area Network).

Note that communication standards other than CAN or next-generation communication standards such as in-vehicle Ethernet or CAN-FD (CAN with Flexible Data rate) may be used for mutual communication between the control units 23A to 23F.

The engine control unit 23A executes engine rotation speed maintenance control to maintain the engine rotation speed at a rotation speed corresponding to the operating position of the accelerator lever, etc., based on the detection information from the accelerator sensor and the detection information from the rotation sensor. do.

The transmission unit control section 23B controls the continuously variable transmission so that the vehicle speed of the tractor 1 is changed to a speed corresponding to the operation position of the transmission lever, based on detection information from the transmission sensor and detection information from the vehicle speed sensor 22A. , and forward/reverse switching control that switches the transmission state of the forward/reverse switching device based on the detection information from the reverser sensor 22B. The vehicle speed control includes a deceleration and stop process in which the continuously variable transmission is decelerated to the zero speed state to stop the tractor 1 from traveling when the speed change lever is operated to the zero speed position.

The work equipment control unit 23D includes a work clutch control that controls the operation of the work clutch unit 19 based on the operation of the PTO switch provided in the drive unit 12, and a control function that controls the operation and height of the lift switch provided in the drive unit 12. Elevation control that controls the operation of the elevating drive unit 20 based on the setting value of the setting dial, and rolling control that controls the operation of the rolling unit 21 based on the setting value of the roll angle setting dial provided in the driving section 12. control, etc. A PTO switch, a lift switch, a height setting dial, and a roll angle setting dial are included in the vehicle condition detection device 22.

As shown in FIG. 6, the tractor 1 is equipped with a positioning unit 30 that measures the position, direction, etc. of the tractor 1. The positioning unit 30 includes a satellite navigation device 31 that measures the position and direction of the tractor 1 using GNSS (Global Navigation Satellite System), which is an example of a satellite positioning system, and a 3-axis gyroscope and a 3-axis gyroscope. An inertial measurement unit (IMU) 32 that has an acceleration sensor and the like and measures the attitude, direction, etc. of the tractor 1 is included. Positioning methods using GNSS include DGNSS (Differential GNSS) and RTK-GNSS (Real Time Kinematic GNSS). In this embodiment, RTK-GNSS suitable for positioning of mobile objects is employed. Therefore, as shown in FIG. 1, a base station 6 that enables positioning by RTK-GNSS is installed at a known position around the field.

As shown in FIGS. 1 and 6, the tractor 1 and the base station 6 each include GNSS antennas 33 and 60 that receive radio waves transmitted from the positioning satellite 7 (see FIG. 1), and the tractor 1 and the base station. Communication modules 34, 61, etc. that enable wireless communication of various information including positioning information with the station 6 are provided. As a result, the satellite navigation device 31 of the positioning unit 30 receives the positioning information obtained by the GNSS antenna 33 of the tractor 1 receiving radio waves from the positioning satellite 7, and the GNSS antenna 60 of the base station 6 receives the radio waves from the positioning satellite 7. The position and direction of the tractor 1 can be measured with high accuracy based on the positioning information obtained by receiving the information. Moreover, the positioning unit 30 can measure the position, azimuth, and attitude angle (yaw angle, roll angle, pitch angle) of the tractor 1 with high precision by having the satellite navigation device 31 and the inertial measurement device 32. .

In this tractor 1, the inertial measurement device 32, GNSS antenna 33, and communication module 34 of the positioning unit 30 are included in the antenna unit 35 shown in FIG. The antenna unit 35 is arranged at the center of the upper left and right sides of the front side of the cabin 13.

Although not shown, the vehicle body position when specifying the position of the tractor 1 is set to the center position of the rear wheel axle. The vehicle body position can be determined from positioning information from the positioning unit 42 and vehicle body information including the positional relationship between the mounting position of the GNSS antenna 45 on the tractor 1 and the center position of the rear wheel axle.

As shown in FIG. 6, the mobile communication terminal 5 includes an electronic control unit in which a microcontroller and the like are integrated, a terminal control unit 51 having various control programs, and the like. The terminal control unit 51 includes a display control section 51A that controls display and notification on the display device 50, a target route generation section 51B that generates a target route P for automatic driving, and a target route generation section 51B that generates a target route P for automatic driving. It includes a non-volatile terminal storage unit 51C that stores the target route P and the like. The terminal storage unit 51C stores, as various information used to generate the target route P, vehicle body information such as the turning radius of the tractor 1, the working width or the number of working ridges of a working device such as the rotary tiller 3, and the above-mentioned information. Field information obtained from positioning information, etc. are stored. In order to identify the shape and size of the field A, the field information includes multiple shape specific points in the field A acquired using GNSS when the tractor 1 is driven along the outer periphery of the field A. (shape specific coordinates), four corner points Cp1 to Cp4 (see Figure 5), and rectangular shape identification to identify the shape and size of field A by connecting these corner points Cp1 to Cp4. The line SL (see FIG. 5), etc. are included.

As shown in FIG. 6, the tractor 1 and the mobile communication terminal 5 include communication modules 28 and 52 that enable wireless communication of various information including positioning information between the on-vehicle control unit 23 and the terminal control unit 51. It is equipped. When Wi-Fi is used for wireless communication with the mobile communication terminal 5, the communication module 28 of the tractor 1 functions as a converter that bidirectionally converts communication information between CAN and Wi-Fi. The terminal control unit 51 can acquire various information regarding the tractor 1 including the position and direction of the tractor 1 through wireless communication with the on-vehicle control unit 23. Thereby, various information including the position and direction of the tractor 1 with respect to the target route P can be displayed on the display device 50 of the mobile communication terminal 5.

The target route generation unit 51B generates a target based on the turning radius of the tractor 1, the working width or the number of working ridges of the working device included in the vehicle body information, the shape and size of the field A included in the field information, etc. Generate route P.

For example, as shown in FIG. 5, in a rectangular farm field A, a start position p1 and an end position p2 of automatic travel are set, and the work traveling direction of the tractor 1 is set in a direction along the short side of the field A. If so, the target path generation unit 51B first converts the field A into a margin area adjacent to the outer periphery of the field A based on the four corner points Cp1 to Cp4 and the rectangular shape specifying line SL. A1 and a workable area A2 located inside the margin area A1.

Next, the target route generation unit 51B sets the workable area A2 at the end of each long side in the workable area A2 based on the turning radius of the tractor 1, the working width of the working device, the number of working ridges, etc. It is divided into a pair of end regions A2a, and a center region A2b, which is set between the pair of end regions A2a. After that, the target route generation unit 51B generates a plurality of parallel routes P1 arranged in parallel at predetermined intervals according to the working width or the number of working ridges in the direction along the long side of the field A in the central area A2b. do. Further, the target route generation unit 51B generates a plurality of connection routes P2 that connect the plurality of parallel routes P1 in the running order of the tractor 1 to each end region A2a.

Thereby, the target route generation unit 51B can generate a target route P that allows the tractor 1 to travel automatically from the automatic travel start position p1 to the end position p2 set in the farm field A shown in FIG. .

In the field A shown in FIG. 5, the margin area A1 is designed to prevent work equipment from coming into contact with other objects such as ridges or fences adjacent to the field A when the tractor 1 automatically travels along the edge of the workable area A2. This is an area secured between the outer periphery of the field A and the workable area A2 in order to prevent this. Each end region A2a is a direction change region when the tractor 1 changes direction from the currently running parallel route P1 toward the next parallel route P1 according to the connection route P2. The central area A2b is a work area in which the tractor 1 automatically travels in a working state along each parallel route P1.

In the target route P shown in FIG. 5, each parallel route P1 is a work route along which the tractor 1 automatically travels while performing work using a work device such as the rotary tiller 3. Each connection route P2 is a non-working route on which the tractor 1 automatically travels without performing any work using the working device. The start position p3 of each parallel path P1 is a work start position where the tractor 1 starts work using the work device. The terminal position p4 of each parallel path P1 is a work stop position where the tractor 1 stops working with the work device. Among the starting positions p3 of each parallel route P1, the starting position p3 of the parallel route P1 in which the running order of the tractor 1 is set first is the starting position p1 of automatic travel. The starting end position p3 of the remaining parallel path P1 is the connection position with the ending position of the connecting path P2. Further, the terminal position p4 of the parallel route P1 in which the running order of the tractor 1 is set last is the end position p2 of automatic running. The terminal position p4 of the remaining parallel path P1 is the connection position with the starting position of the connection path P2.

It should be noted that the target route P shown in FIG. Based on field information such as shape and size, various target routes P suitable for them can be generated.

The target route P is stored in the terminal storage unit 51C in a state in which it is associated with vehicle body information, field information, etc., and can be displayed on the display device 50 of the mobile communication terminal 5. The target route P includes the target vehicle speed of the tractor 1 on each parallel route P1, the target vehicle speed of the tractor 1 on each connection route P2, the front wheel steering angle on each parallel route P1, the front wheel steering angle on each connection route P2, etc. include.

The terminal control unit 51 transmits the field information, target route P, etc. stored in the terminal storage section 51C to the on-board control unit 23 in response to a transmission request command from the on-board control unit 23. The on-vehicle control unit 23 stores the received field information, target route P, etc. in the on-vehicle storage section 23G. Regarding the transmission of the target route P, for example, the terminal control unit 51 transmits the entire target route P from the terminal storage section 51C to the on-vehicle control unit 23 at once at a stage before the tractor 1 starts automatic travel. You may also do so. Further, the terminal control unit 51 divides the target route P into a plurality of divided route information for each predetermined distance, and each time the travel distance of the tractor 1 reaches a predetermined distance from a stage before the tractor 1 starts automatic travel. , a predetermined number of divided route information according to the running order of the tractor 1 may be sequentially transmitted from the terminal storage section 51C to the on-vehicle control unit 23.

In the in-vehicle control unit 23, detection information from various sensors and switches included in the vehicle condition detection device 22 is input to the automatic driving control section 23F via the transmission unit control section 23B, the steering control section 23C, etc. has been done. Thereby, the automatic travel control unit 23F can monitor various setting states and operating states of each part in the tractor 1.

The automatic travel control unit 23F changes the travel mode of the tractor 1 from manual travel mode to automatic travel mode when various manual setting operations are performed by a user such as a passenger or administrator to satisfy automatic travel start conditions. In the switched state, when the display device 50 of the mobile communication terminal 5 is operated to instruct the start of automatic driving, the tractor 1 follows the target route P while acquiring the position and direction of the tractor 1 using the positioning unit 30. 1 starts automatic travel control to automatically travel.

The automatic driving control unit 23F is activated when, for example, a user operates the display device 50 of the mobile communication terminal 5 to instruct the end of automatic driving while the automatic driving control is being executed, or when the user is on board the driving unit 12. When the user operates a manual operating tool such as the steering wheel 25 or an accelerator pedal, the automatic driving control is ended and the driving mode is switched from the automatic driving mode to the manual driving mode.

The automatic travel control by the automatic travel control unit 23F includes an automatic engine control process that transmits a control command for automatic travel regarding the engine 14 to the engine control unit 23A, and automatic travel control regarding the vehicle speed of the tractor 1 and switching between forward and backward movement. Automatic control processing for vehicle speed that transmits commands to the transmission unit control section 23B, automatic control processing for steering that transmits control commands for automatic driving related to steering to the steering control section 23C, and automatic control processing regarding working devices such as the rotary tiller 3. It includes automatic control processing for work that transmits control commands for traveling to the work equipment control unit 23D, and the like.

In the engine automatic control process, the automatic travel control unit 23F issues an engine rotation speed change command to the engine control unit 23A to instruct the engine rotation speed to be changed based on the set rotation speed included in the target route P. Send. The engine control unit 23A executes engine rotation speed change control that automatically changes the engine rotation speed in response to various control commands regarding the engine 14 transmitted from the automatic travel control unit 23F.

In the vehicle speed automatic control process, the automatic driving control unit 23F issues a speed change operation command that instructs the continuously variable transmission to perform a speed change operation based on the target vehicle speed included in the target route P, and A forward/reverse switching command for instructing a forward/reverse switching operation of the forward/reverse switching device based on the traveling direction of the tractor 1, etc. is transmitted to the transmission unit control section 23B. The transmission unit control section 23B performs vehicle speed control that automatically controls the operation of the continuously variable transmission in response to various control commands related to the continuously variable transmission, forward/reverse switching device, etc. transmitted from the automatic driving control section 23F; It also performs automatic forward/reverse switching control that automatically controls the operation of the forward/reverse switching device. The vehicle speed control includes, for example, an automatic deceleration and stop process that controls the continuously variable transmission to decelerate to a zero speed state to stop the tractor 1 from traveling when the target vehicle speed included in the target route P is zero speed. It is included.

In the automatic steering control process, the automatic travel control unit 23F transmits a steering command for instructing the steering of the left and right front wheels 10 based on the front wheel steering angle included in the target route P, etc. to the steering control unit 23C. . The steering control unit 23C performs automatic steering control that controls the operation of the power steering unit 17 to steer the left and right front wheels 10 in response to a steering command transmitted from the automatic travel control unit 23F, and controls the settings of the left and right front wheels 10. When the vehicle is steered beyond the angle, automatic brake turning control is executed in which the brake unit 18 is activated to activate the brake on the inside of the turn.

In the work automatic control process, the automatic travel control unit 23F controls the rotary tiller 3, etc. based on the arrival of the tractor 1 at each work start position included in the target route P (the starting position p3 of each parallel route P1). The work device is activated based on the work start command that instructs the switching of the work device to the working state, and the arrival of the tractor 1 at each work stop position included in the target path P (the terminal position p4 of each parallel path P1). A work stop command that instructs switching to a non-working state, etc. is transmitted to the work equipment control unit 23D. The work equipment control unit 23D controls the operation of the lifting drive unit 20 and the like in accordance with various control commands related to the work equipment transmitted from the automatic travel control unit 23F, and lowers the work equipment to a working height to operate the work equipment. Executes automatic work start control, automatic work stop control that raises the work equipment to a non-work height and puts it on standby, etc.

In other words, the automatic driving unit 4 described above includes a power steering unit 17, a brake unit 18, a working clutch unit 19, an elevation drive unit 20, a rolling unit 21, a vehicle state detection device 22, an on-vehicle control unit 23, a positioning unit 30, and , communication modules 28, 34, and the like. By operating these properly, the tractor 1 can be automatically driven along the target route P with high accuracy, and work can be performed properly using a working device such as the rotary tiller 3.

As shown in FIGS. 6 and 7, the tractor 1 is equipped with an obstacle detection system 80 that monitors the surroundings of the tractor 1 and detects obstacles existing around the tractor 1. Obstacles detected by the obstacle detection system 80 include people such as workers working in the field A, other work vehicles, and utility poles and trees existing in the field A.

As shown in FIG. 7, the obstacle detection system 80 includes an imaging unit 80A that captures visible light around the tractor to obtain color image information, and an obstacle detection unit that detects obstacles that exist around the tractor 1. 80B, and an information integration processing section 80C that integrates and processes color image information from the imaging unit 80A and detection information from the obstacle detection unit 80B.

As shown in FIGS. 1 to 3 and 7, the imaging unit 80A includes a front camera 81 having a first imaging range Ri1 in front of the cabin 13 as the imaging range, and a second imaging range Ri2 in the rear from the cabin 13. After the camera 82 was set as the imaging range, the right camera 83 set the third imaging range Ri3 to the right from the cabin 13 as the imaging range, and the fourth imaging range Ri4 to the left from the cabin 13 was set as the imaging range. It includes a left camera 84 and an image processing device 85 (see FIG. 7) that processes color image information from each of the cameras 81 to 84.

The front camera 81 and the rear camera 82 are arranged on the left and right center line of the tractor 1. The front camera 81 is disposed at the left-right center of the upper part of the front end side of the cabin 13 in a forward-sloping posture looking down on the front side of the tractor 1 from diagonally above. Thereby, in the front camera 81, a predetermined range on the front side of the vehicle body with the left-right center line of the tractor 1 as the axis of symmetry is set as the first imaging range Ri1. The rear camera 82 is disposed at the left-right center of the upper part on the rear end side of the cabin 13 in a backward-sloping position looking down on the rear side of the tractor 1 from diagonally above. Thereby, in the rear camera 82, a predetermined range on the rear side of the vehicle body with the left-right center line of the tractor 1 as the axis of symmetry is set as the second imaging range Ri2. The right camera 83 is disposed at the front-rear center of the upper part on the right end side of the cabin 13 in a downward-sloping position looking down on the right side of the tractor 1 from diagonally above. As a result, the right camera 83 has a predetermined range on the right side of the vehicle body set as the third imaging range Ri3. The left camera 84 is disposed at the front-rear center of the upper part on the left end side of the cabin 13 in a left-down position looking down on the left side of the tractor 1 from diagonally above. As a result, the left camera 84 has a predetermined range on the left side of the vehicle body set as the fourth imaging range Ri4.

The image processing device 85 is constructed from an electronic control unit in which a microcontroller and the like are integrated, various control programs, and the like. The image processing device 85 is connected to the in-vehicle control unit 23, the information integration processing section 80C, and the like via CAN so as to be able to communicate with each other.

The image processing device 85 performs image processing on color image information sequentially transmitted from each camera 81 to 84. For example, the image processing device 85 performs an image generation process of generating front, rear, left, and right images corresponding to the imaging range of each camera 81 to 84 for color image information sequentially transmitted from each camera 81 to 84, and An all-around image generation process is performed in which color image information from all cameras 81 to 84 is combined to generate an all-around image (for example, a surround view) of the tractor 1. Then, image transmission processing is performed to transmit each generated image and the entire surrounding image to the display control section 23E of the on-vehicle control unit 23. The display control unit 23E transmits each image and the entire surrounding image from the image processing device 85 to the operating terminal 27 via CAN, and also to the display control unit 51A of the mobile communication terminal 5 via the communication modules 28 and 52. Send.

Thereby, the all-around image generated by the image processing device 85, the image corresponding to the traveling direction of the tractor 1, etc. can be displayed on the operation terminal 27 of the tractor 1, the display device 50 of the mobile communication terminal 5, etc. This display allows the user to visually recognize the surrounding situation and the running direction of the tractor 1.

In other words, the display control section 23E of the on-vehicle control unit 23 displays the all-around image generated by the image processing device 85, the image corresponding to the traveling direction of the tractor 1, etc. on the display device of the operation terminal 27 of the tractor 1 and the mobile communication terminal 5. 50 etc., it functions as a work support unit that enables work support such as making it easier for the user to visually recognize the surrounding situation of the work vehicle V.

The image processing device 85 is subjected to learning processing to recognize people such as workers working in the field A, other work vehicles, and electric poles, trees, etc. existing in the field A as obstacles. . Thereby, the image processing device 85 detects obstacles that affect the running of the tractor 1 in any of the imaging ranges Ri1 to Ri4 of each of the cameras 81 to 84 based on the color image information sequentially transmitted from each of the cameras 81 to 84. Obstacle determination processing can be performed to determine whether an object exists.

In the obstacle determination process, if it is determined that an obstacle exists in any of the imaging ranges Ri1 to Ri4, the image processing device 85 performs a coordinate calculation process to calculate the coordinates of the obstacle on the image where the obstacle exists. Based on the mounting position and mounting angle of each camera 81 to 84, a coordinate conversion process is performed to convert the obtained coordinates of the obstacle into coordinates based on the vehicle body coordinate origin. Then, a distance calculation process is performed to calculate the straight line distance between the converted coordinates and a preset distance calculation reference point as the distance from the distance calculation reference point to the obstacle, and the distance between the converted coordinates and the calculated obstacle is calculated. Obstacle information transmission processing is performed to transmit the distance to the information integration processing unit 80C as information regarding the obstacle. On the other hand, if there is no obstacle in any of the imaging ranges Ri1 to Ri4, an undetected transmission process is performed to transmit the fact that no obstacle has been detected to the information integration processing unit 80C.

In this way, if an obstacle exists in any of the imaging ranges Ri1 to Ri4 of the cameras 81 to 84, the image processing device 85 transmits information regarding the obstacle to the information integration processing section 80C. By receiving the information regarding the obstacle, the integrated processing unit 80C is able to grasp the existence of the obstacle in the imaging range Ri1 to Ri4 of each camera 81 to 84, and also determine the position of the obstacle. and the distance to the obstacle can be obtained. Furthermore, if there is no obstacle in any of the imaging ranges Ri1 to Ri4 of the cameras 81 to 84, the image processing device 85 sends the information that no obstacle has been detected to the information integration processing section 80C. By receiving the undetected information, the processing unit 80C can grasp that no obstacle exists in any of the imaging ranges Ri1 to Ri4 of the cameras 81 to 84.

The vehicle body coordinate origin in the above coordinate conversion process and the distance calculation reference point in the distance calculation process are set according to the mounting position of each camera 81 to 84. Specifically, as shown in FIG. 8, a vehicle body coordinate origin O1 and a distance calculation reference point Rp1 are set for the front camera 81 according to its mounting position. For the rear camera 82, a vehicle body coordinate origin O2 and a distance calculation reference point Rp2 are set according to its mounting position. For the right camera 83, a vehicle body coordinate origin O3 and a distance calculation reference point Rp3 are set according to its mounting position. For the left camera 84, a vehicle body coordinate origin O4 and a distance calculation reference point Rp4 are set according to its mounting position.

Thereby, for example, if an obstacle exists in the first imaging range Ri1 of the front camera 81, the image processing device 85 calculates the coordinates of the obstacle on the image of the front camera 81 where the obstacle exists (coordinates calculation process), and converts the obtained coordinates of the obstacle into coordinates (x, y) based on the vehicle body coordinate origin O1 shown in FIG. 8 based on the mounting position and mounting angle of the front camera 81 (coordinate conversion The straight line distance between the converted coordinates (x, y) and the distance calculation reference point Rp1 is determined as the distance La from the distance calculation reference point Rp1 to the obstacle O (distance calculation process).

Note that the relationships among the vehicle body coordinate origins O1 to O4, the distance calculation reference points Rp1 to Rp4, and the mounting positions of the cameras 81 to 84 described above can be changed in various settings.

As shown in FIG. 1, FIGS. 3 to 4, and FIG. A second obstacle sensor 87 that detects obstacles and a third obstacle sensor 88 that detects obstacles on the left and right sides of the tractor 1 are included. The first obstacle sensor 86 and the second obstacle sensor 87 employ lidar sensors that use pulsed near-infrared laser light (an example of near-infrared light) to detect obstacles. The third obstacle sensor 88 employs sonar that uses ultrasonic waves to detect obstacles.

As shown in FIGS. 1 to 4 and FIG. 7, the first obstacle sensor 86 includes a first measurement unit 86A that measures the distance to a measurement object existing in the measurement range using near-infrared laser light; It has a first control section 86B that generates a distance image based on the measurement information from the first measurement section 86A. The second obstacle sensor 87 includes a second measuring section 87A that measures the distance to the object to be measured existing in the measurement range using near-infrared laser light, and a second obstacle sensor 87 based on the measurement information from the second measuring section 87A. The second controller 87B includes a second controller 87B that performs operations such as generating a distance image. The third obstacle sensor 88 exists in the measurement range based on the ultrasonic sensor 88A on the right and the ultrasonic sensor 88B on the left that transmit and receive ultrasonic waves, and the ultrasonic sensors 88A and 88B that transmit and receive ultrasonic waves. It has a single third control unit 88C that measures the distance to the object to be measured.

The control units 86B, 87B, and 88C of each of the obstacle sensors 86 to 88 are constructed by an electronic control unit in which a microcontroller and the like are integrated, various control programs, and the like. Each of the control sections 86B, 87B, and 88C is connected to the on-vehicle control unit 23, the information integration processing section 80C, and the like via CAN so as to be able to communicate with each other.

As shown in FIGS. 3 and 4, the first obstacle sensor 86 is set to have a first measurement range Rm1 in front of the cabin 13 as its measurement range. The second obstacle sensor 87 has a second measurement range Rm2 rearward from the cabin 13 set as its measurement range. The third obstacle sensor 88 has a third measurement range Rm3 to the right of the cabin 13 and a fourth measurement range Rm4 to the left of the cabin 13 as measurement ranges.

As shown in FIGS. 1 and 3 to 4, the first obstacle sensor 86 and the second obstacle sensor 87 are arranged on the left-right center line of the tractor 1, similarly to the front camera 81 and the rear camera 82. The first obstacle sensor 86 is disposed at the left-right center of the upper part of the front end side of the cabin 13 in a forward-down position looking down on the front side of the tractor 1 from diagonally above. Thereby, in the first obstacle sensor 86, a predetermined range on the front side of the vehicle body with the left-right center line of the tractor 1 as the axis of symmetry is set as the first measurement range Rm1 by the measurement unit 86A. The second obstacle sensor 87 is disposed at the left-right center of the upper part on the rear end side of the cabin 13 in a backward-sloping posture looking down on the rear side of the tractor 1 from diagonally above. Thereby, in the second obstacle sensor 87, a predetermined range on the rear side of the vehicle body with the left-right center line of the tractor 1 as the axis of symmetry is set as the second measurement range Rm2 by the measurement unit 87A.

When the tractor 1 is traveling forward with the forward/reverse switching device of the transmission unit 16 switched to the forward transmission state, the first obstacle sensor 86 and the second obstacle sensor 87 are connected to the first obstacle sensor 87 in conjunction with the switching. 86 is activated, and the second obstacle sensor 87 is deactivated. Further, when the tractor 1 is traveling in reverse with the forward/reverse switching device of the transmission unit 16 switched to the reverse transmission state, the first obstacle sensor 86 becomes deactivated in conjunction with the switching, and the second obstacle sensor 87 becomes operational.

As shown in FIG. 2, the right ultrasonic sensor 88A is attached to the right ingress/egress step 24 disposed between the right front wheel 10 and the right rear wheel 11 so as to face outward to the right of the vehicle body. As a result, the right ultrasonic sensor 88A has a predetermined range on the right outer side of the vehicle body set as the third measurement range Rm3. As shown in FIGS. 1 to 3, the left ultrasonic sensor 88B is attached to the left ingress/egress step 24 located between the left front wheel 10 and the left rear wheel 11 so as to face outward to the left of the vehicle body. . As a result, the left ultrasonic sensor 88B has a predetermined range on the left outer side of the vehicle body set as the fourth measurement range Rm4.

As shown in FIGS. 3 to 4 and FIG. 7, the measurement units 86A and 87A of the first obstacle sensor 86 and the second obstacle sensor 87 allow the irradiated near-infrared laser light to reach the distance measurement point and return. Using the TOF (Time of Flight) method, which measures the distance to the distance measurement point based on the round trip time, each distance measurement point (measurement point) in the first measurement range Rm1 or the second measurement range Rm2 is Measure the distance to the object (example). Each measurement unit 86A, 87A scans the near-infrared laser beam vertically and horizontally at high speed over the entire first measurement range Rm1 or second measurement range Rm2, and measures the distance to the distance measurement point for each scanning angle (coordinate). By sequentially measuring , three-dimensional measurement is performed in the first measurement range Rm1 or the second measurement range Rm2. Each measuring section 86A, 87A measures the intensity (hereinafter referred to as "intensity") of reflected light from each distance measurement point obtained when near-infrared laser light is scanned vertically and horizontally at high speed over the entire first measurement range Rm1 or second measurement range Rm2. , referred to as reflection intensity) are sequentially measured. Each measuring section 86A, 87A repeatedly measures the distance to each distance measurement point, each reflection intensity, etc. in the first measurement range Rm1 or the second measurement range Rm2 in real time.

Each control section 86B, 87B of the first obstacle sensor 86 and the second obstacle sensor 87 controls the distance to each ranging point measured by each measuring section 86A, 87A, the scanning angle (coordinates) for each ranging point, etc. From the measurement information, a distance image is generated and a group of distance measurement points estimated to be obstacles is extracted, and measurement information regarding the extracted distance measurement point group is transmitted to the information integration processing unit 80C as measurement information regarding the obstacle candidate. .

Each control section 86B, 87B of the first obstacle sensor 86 and second obstacle sensor 87 determines whether the distance value of each ranging point measured by each measuring section 86A, 87A satisfies an invalid condition. , the distance value that meets the invalid conditions is sent to the information integration processing unit 80C as an invalid value.

Specifically, each control unit 86B, 87B has the characteristic by utilizing the characteristic of dirt on the sensor surface that is present at a close distance from the first obstacle sensor 86 or the second obstacle sensor 87. The distance value of the distance measurement point is set as an invalid value. This prevents the distance value of the distance measurement point regarding dirt on the sensor surface from being used as measurement information regarding the obstacle in the information integration processing section 80C.

Furthermore, each of the control units 86B and 87B utilizes the characteristic of floating objects such as dust and fog that they exist close to the first obstacle sensor 86 or the second obstacle sensor 87 but have a very weak reflection intensity. , the distance value of the distance measurement point having the characteristic is set as an invalid value. This prevents the distance value of the distance measurement point regarding the floating object from being used as measurement information regarding the obstacle in the information integration processing section 80C.

As shown in FIGS. 3 to 4 and FIG. 7, the third control unit 88C of the third obstacle sensor 88 controls the third measurement range Rm3 or the third measurement range Rm3 or 4. The presence or absence of the measurement target object in the measurement range Rm4 is determined. The third control unit 88C controls each ultrasonic sensor 88A using the TOF (Time of Flight) method, which measures the distance to the distance measurement point based on the round trip time for the transmitted ultrasonic wave to reach the distance measurement point and return. , 88B to the object to be measured, and transmits the measured distance to the object to be measured and the direction of the object to be measured as measurement information regarding the obstacle to the information integration processing section 80C.

As shown in FIGS. 4 and 9 to 11, each control section 86B, 87B of the first obstacle sensor 86 and second obstacle sensor 87 controls the vehicle body relative to the measurement range Rm1, Rm2 of each measurement section 86A, 87A. By performing cut processing and masking processing based on information etc., the range of obstacles detected by the first obstacle sensor 86 and the second obstacle sensor 87 is limited to the first detection range Rd1 and the second detection range Rd2. are doing.

In the cutting process, each control section 86B, 87B acquires the maximum horizontal width of the vehicle body including the working device (in this embodiment, the horizontal width of the rotary tiller 3) through communication with the vehicle-mounted control unit 23, and The obstacle detection target width Wd is set by adding a predetermined safety band to the maximum horizontal width. Then, in the first measurement range Rm1 and the second measurement range Rm2, the left and right ranges that deviate from the detection target width Wd are set as the first non-detection range Rnd1 by cutting processing and excluded from each detection range Rd1, Rd2.

In the masking process, each control section 86B, 87B sets a predetermined range in which the front end of the tractor 1 enters into the first measurement range Rm1, and into a range into which the rear end of the working device enters into the second measurement range Rm2. The range including the safety band is set as a second non-detection range Rnd2 by masking processing and excluded from each detection range Rd1, Rd2.

By limiting the obstacle detection range to the first detection range Rd1 and the second detection range Rd2 in this way, the first obstacle sensor 86 and the second obstacle sensor 87 can An increase in the detection load due to detecting an obstacle that has come off and has no risk of colliding with the tractor 1, or the front end side of the tractor 1 or the rear end of the working device that has entered the first measurement range Rm1 or the second measurement range Rm2. This avoids the possibility of erroneously detecting the side as an obstacle.

Note that the second non-detection range Rnd2 shown in FIG. 9 is an example of a non-detection range suitable for the front side of the vehicle body where the left and right front wheels 10 and the bonnet 15 are present. The second non-detection range Rnd2 shown in FIG. 10 is an example of a non-detection range suitable for a working state in which the rotary tiller 3 is lowered to the working height on the rear side of the vehicle body. The second non-detection range Rnd2 shown in FIG. 11 is an example of a non-detection range suitable for a non-working state in which the rotary tiller 3 is raised to the retracted height on the rear side of the vehicle body. The second non-detection range Rnd2 on the rear side of the vehicle body is appropriately switched in conjunction with the vertical movement of the rotary tiller 3.

Information regarding the first detection range Rd1, the second detection range Rd2, the first non-detection range Rnd1, and the second non-detection range Rnd2 is included in the above-mentioned distance image, and together with the above-mentioned distance image, the information integration processing unit It is being sent to 80C.

As shown in FIG. 4, the detection ranges Rd1 and Rd2 of the first obstacle sensor 86 and the second obstacle sensor 87 are determined based on the collision determination process in which the predicted collision time reaches a set time (for example, 3 seconds). It is divided into a control range Rsc, a deceleration control range Rdc, and a notification control range Rnc. The stop control range Rsc is set to the range from the first obstacle sensor 86 or the second obstacle sensor 87 to the determination reference position for collision determination processing. The deceleration control range Rdc is set from the determination reference position to the deceleration start position. The notification control range Rnc is set from the deceleration start position to the measurement limit position of the first obstacle sensor 86 or the second obstacle sensor 87. Each determination reference position is set at a position that is a certain distance L (for example, 2000 mm) from the front end or rear end of the vehicle body including the rotary tilling device 3 in the longitudinal direction of the vehicle body.

As shown in FIG. 7, the information integration processing section 80C is constructed from an electronic control unit in which a microcontroller and the like are integrated, various control programs, and the like. The information integration processing unit 80C combines information from the imaging unit 80A, which has low distance measurement accuracy but high object discrimination accuracy, and measurement information from the obstacle detection unit 80B, which has low object discrimination accuracy but high distance measurement accuracy. Obstacle information output control is executed to acquire and output information regarding obstacles based on the information.

Hereinafter, the control operation of the information integration processing section 80C in the obstacle information acquisition control will be explained based on the flowchart shown in FIG. 12.
Note that here, the information integration processing section 80C obtains information regarding the obstacle based on color image information from the front camera 81 of the imaging unit 80A and measurement information from the first obstacle sensor 86 of the obstacle detection unit 80B. Only the control operation when acquiring and outputting will be explained. Other than this, when obtaining and outputting information regarding an obstacle based on information from the rear camera 82 of the imaging unit 80A and measurement information from the second obstacle sensor 87 of the obstacle detection unit 80B, and , information integration when obtaining and outputting information regarding an obstacle based on information from the left and right cameras 83, 84 of the imaging unit 80A and measurement information from the third obstacle sensor 88 of the obstacle detection unit 80B. Since the control operation of the processing section 80C is also the same as the case based on the information from the front camera 81 and the measurement information from the first obstacle sensor 86, the following describes the control operation of the information integration processing section 80C in these cases. is omitted.

The information integration processing section 80C performs a first determination process of determining whether an obstacle is detected from the color image information of the front camera 81 based on the information from the imaging unit 80A (step #1), and A second determination process is performed to determine whether or not an obstacle candidate is included in the distance image of the first obstacle sensor 86 based on the measurement information from the obstacle detection unit 80B (step #2).

The information integration processing unit 80C detects that an obstacle is detected from the color image information of the front camera 81 in the first determination process, and that an obstacle candidate is included in the distance image of the first obstacle sensor 86 in the second determination process. If so, a third determination process is performed to determine whether or not the position of the aforementioned obstacle matches the position of the obstacle candidate (step #3).

If the position of the obstacle and the position of the obstacle candidate match in the third determination process, the information integration processing unit 80C detects the obstacle (obstacle candidate) by the front camera 81 and the first obstacle sensor 86. It is determined that the detection state is a proper detection state in which the front camera 81 and the first obstacle sensor 86 are properly detecting the same obstacle (obstacle candidate). Based on this determination result, the distance information of the obstacle candidate measured by the first obstacle sensor 86 is applied to the obstacle detected from the color image information of the front camera 81, and this information is used for obstacle detection. Appropriate information acquisition processing for acquiring information and appropriate information output processing for outputting the acquired obstacle detection information are performed (steps #4 to #5).

If the position of the obstacle and the position of the obstacle candidate do not match, the information integration processing unit 80C determines the position of the obstacle in the color image information of the front camera 81 based on the distance information from the first obstacle sensor 86. A fourth determination process is performed to determine whether the measured value at the position corresponding to is valid (step #6).

If the measurement value described above is valid in the fourth determination process, the information integration processing unit 80C determines that the detection state of the obstacle (obstacle candidate) by the front camera 81 and the first obstacle sensor 86 is the same as that of the front camera 81. Although the obstacle is properly detected from the color image information and the measurement by the first obstacle sensor 86 is properly performed, the distance image of the first obstacle sensor 86 shows that the obstacle is not an obstacle. It is determined that the state is in a semi-appropriate detection state where it cannot be specified as a candidate. Based on this determination result, the first obstacle sensor 86 measures the obstacle detected from the color image information of the front camera 81 with respect to the measurement target corresponding to the position of the obstacle indicated by the front camera 81. Applying the acquired distance information, a semi-appropriate information acquisition process of acquiring this information as obstacle detection information and a semi-appropriate information output process of outputting the acquired obstacle detection information are performed (steps #7 to #8).

If the measurement value described above is not valid in the fourth determination process, the information integration processing unit 80C determines that the detection state of the obstacle (obstacle candidate) by the front camera 81 and the first obstacle sensor 86 is the same as that of the front camera 81. Obstacles are properly detected from the color image information, but in the distance image of the first obstacle sensor 86, dust, fog, etc. are detected at the measurement target position corresponding to the position of the obstacle indicated by the front camera 81. The measurement value for the object to be measured corresponding to the position of the obstacle indicated by the front camera 81 is invalid due to the occurrence of floating objects or the adhesion of dirt on the sensor surface of the first obstacle sensor 86.Invalid measurement value. It is determined that the state is the same. Based on this determination result, the distance information for the obstacle calculated from the color image information of the front camera 81 is applied to the obstacle detected from the color image information of the front camera 81, and this information is used for obstacle detection. A camera-only information acquisition process for acquiring information and a camera-only information output process for outputting the acquired obstacle detection information are performed (steps #9 and #10).

The on-vehicle control unit 23 includes an obstacle control section 23H that executes control regarding obstacles based on obstacle detection information from the information integration processing section 80C. The obstacle control section 23H is constructed by an electronic control unit in which a microcontroller and the like are integrated, various control programs, and the like. The obstacle control section 23H is connected to the information integration processing section 80C and the like via CAN so as to be able to communicate with each other.

The obstacle control section 23H executes collision avoidance control to avoid collision with an obstacle based on the obstacle detection information from the information integration processing section 80C. Specifically, when the obstacle control unit 23H acquires the obstacle detection information through the appropriate information output processing or the semi-appropriate information output processing from the information integration processing unit 80C, the obstacle control unit 23H performs the first collision avoidance control as the collision avoidance control. Execute. When the obstacle control unit 23H acquires the obstacle detection information through the camera-only information output process from the information integration processing unit 80C, the obstacle control unit 23H performs a second collision avoidance control, which has a higher collision avoidance rate than the first collision avoidance control, as a collision avoidance control. Executes collision avoidance control.

Hereinafter, the control operation of the obstacle control section 23H in the first collision avoidance control will be explained based on the flowchart shown in FIG. 13.
Note that here, a case where an obstacle is located in the first detection range Rd1 of the first obstacle sensor 86 will be explained as an example.

The obstacle control unit 23H determines whether the obstacle is located in the notification control range Rnc of the first detection range Rd1 shown in FIG. 4 based on the distance to the obstacle included in the obstacle detection information. a fifth determination process that determines whether or not the vehicle is located within the deceleration control range Rdc; a seventh determination process that determines whether the vehicle is located within the stop control range Rsc; (Steps #11 to #13).

If the obstacle control unit 23H detects in the fifth determination process that the obstacle is located in the notification control range Rnc of the first detection range Rd1, the obstacle control unit 23H determines that the obstacle is located in the notification control range Rnc. A first command for instructing the display control section 23E of the on-vehicle control unit 23 and the display control section 51A of the terminal control unit 51 to issue a notification command for making a notification on the operation terminal 27 of the tractor 1 or the display device 50 of the mobile communication terminal 5. Notification command processing is performed (step #14).

Thereby, it is possible to notify users such as the occupant of the driving section 12 and the administrator outside the vehicle that an obstacle is located in the notification control range Rnc of the first detection range Rd1 for the tractor 1.

If the obstacle control unit 23H detects in the sixth determination process that the obstacle is located in the deceleration control range Rdc of the first detection range Rd1, the obstacle control unit 23H determines that the obstacle is located in the deceleration control range Rdc. A second control unit that instructs the display control unit 23E of the on-vehicle control unit 23 and the display control unit 51A of the terminal control unit 51 to issue a notification command to the operation terminal 27 of the tractor 1 and the display device 50 of the mobile communication terminal 5. Notification command processing is performed (step #15). The obstacle control unit 23H also performs deceleration command processing to instruct the transmission unit control unit 23B to issue a deceleration command to reduce the vehicle speed of the tractor 1 as the obstacle located in the deceleration control range Rdc approaches the tractor 1. (Step #16).

Thereby, it is possible to notify users such as the occupant of the driving section 12 and the administrator outside the vehicle that an obstacle is located in the deceleration control range Rdc of the first detection range Rd1 for the tractor 1. Furthermore, the speed of the tractor 1 can be appropriately reduced as the work vehicle V approaches an obstacle by the control operation of the transmission unit control section 23B.

If the obstacle control unit 23H detects in the seventh determination process that the obstacle is located in the stop control range Rsc of the first detection range Rd1, the obstacle control unit 23H determines that the obstacle is located in the stop control range Rsc. A third control unit that instructs the display control unit 23E of the on-vehicle control unit 23 and the display control unit 51A of the terminal control unit 51 to issue a notification command to the operation terminal 27 of the tractor 1 and the display device 50 of the mobile communication terminal 5. Notification command processing is performed (step #17). The obstacle control unit 23H also performs a deceleration and stop command process to instruct the transmission unit control unit 23B to decelerate and stop the tractor 1 while the obstacle is located within the stop control range Rsc (step #18).

Thereby, it is possible to notify users such as the occupant of the driving section 12 and the administrator outside the vehicle that an obstacle is located within the stop control range Rsc of the first detection range Rd1 for the tractor 1. Furthermore, the control operation of the transmission unit control section 23B allows the tractor 1 to be decelerated and stopped while the obstacle is located within the stop control range Rsc, thereby avoiding the possibility that the work vehicle V will collide with the obstacle. I can do it.

Next, the control operation of the obstacle control section 23H in the second collision avoidance control will be explained. The deceleration control range is set so that the distance from the determination reference position to the boundary between the deceleration control range Rdc and the stop control range Rsc and the distance from the determination reference position to the boundary between the notification control range Rnc and the deceleration control range Rdc become longer. The same control processes as steps #11 to #18 in the first collision avoidance control are performed with the longitudinal lengths of Rdc and stop control range Rsc being increased. Thereby, the second collision avoidance control is a collision avoidance control that can avoid a collision with an obstacle at an earlier timing than the first collision avoidance control.

Thereby, the obstacle control unit 23H performs second collision avoidance control to avoid a collision with an obstacle based on the measurement information from the imaging unit 80A, which has lower ranging accuracy than the measurement information from the obstacle detection unit 80B. In this case, a collision with an obstacle can be avoided with a higher collision avoidance rate than in the first collision avoidance control. As a result, collisions with obstacles can be effectively avoided based on the measurement information from the imaging unit 80A, which has low ranging accuracy.

The automatic travel control unit 23F performs automatic travel control based on the adhesion of dirt, etc. when it detects the adhesion of dirt (measurement obstruction) to the sensor surfaces of the first obstacle sensor 86 and the second obstacle sensor 87. This includes dirt handling control processing that controls the running of the tractor 1.

Hereinafter, the control operation of the automatic travel control section 23F in the dirt handling control process will be explained based on the flowchart shown in FIG. 14.
Note that the case of the first obstacle sensor 86 will also be described here as an example.

The automatic driving control unit 23F determines the ratio of invalid values caused by measurement obstacles such as dirt included in the measurement information from the first obstacle sensor 86 to the measurement range Rm1 of the first obstacle sensor 86. An eighth determination process is performed to determine whether or not is equal to or greater than a predetermined value (for example, 50%) (step #21).

If the percentage occupied by the invalid value is equal to or higher than a predetermined value in the eighth determination process, the automatic travel control unit 23F sets the vehicle speed of the tractor 1 to a speed higher than that capable of maintaining the traveling state of the tractor 1 in the creep traveling state. A deceleration command process is performed in which a deceleration command for reducing the speed to a low speed is commanded to the transmission unit control section 23B (step #22).

If the percentage occupied by the invalid value is not equal to or higher than the predetermined value in the eighth determination process, the automatic travel control unit 23F issues a vehicle speed maintenance command to the transmission unit control unit 23B to maintain the vehicle speed of the tractor 1 at the current vehicle speed. A vehicle speed maintenance command process is executed (step #23).

After performing the deceleration command process, the automatic travel control unit 23F performs a ninth determination process to determine whether the creep travel state of the tractor 1 continues for a predetermined period of time (step #24). Then, the automatic driving control unit 23F performs the above-mentioned eighth determination process until the creep driving state continues for a predetermined time (step #25), and in this eighth determination process, the proportion occupied by invalid values becomes a predetermined value. If the value falls below this value, it is not because dirt has adhered to the sensor surface of the first obstacle sensor 86, but because there is floating matter such as dust or mist floating around the first obstacle sensor 86. When it is determined that the vehicle speed is present, a vehicle speed return command process is performed to instruct the transmission unit control section 23B to issue a vehicle speed return command to return the vehicle speed of the tractor 1 to the original vehicle speed before the vehicle speed was reduced to an extremely low speed (step #26). , and then returns to step #21.

If the creep driving state continues for a predetermined period of time, the automatic travel control unit 23F determines that dirt has adhered to the sensor surface of the first obstacle sensor 86, and immediately stops the tractor 1 from traveling. A traveling stop command process is performed to issue a stop command to the speed change unit control section 23B (step #27).

In other words, during automatic travel of the tractor 1 under the automatic travel control of the automatic travel control unit 23F, if the ratio of invalid values to the measurement range Rm1 of the first obstacle sensor 86 exceeds a predetermined value, the tractor The vehicle speed of tractor 1 is reduced to a very low speed, which is slower than the low speed, and the running state of tractor 1 is maintained in a creep running state. The time required to determine whether the cause of the value is dirt or other substances adhering to the sensor surface of each obstacle sensor 86 to 88, or floating objects such as dust or dirt that have been kicked up around each obstacle sensor 86 to 88. can be made longer.

By lengthening the determination time in this way, it becomes easier to determine whether the cause of the invalid value is attached matter or floating matter. Therefore, it is possible to suppress a decrease in work efficiency due to the tractor 1 stopping traveling. Moreover, it is possible to suppress the occurrence of an inconvenience in which the work vehicle V collides with an obstacle while determining whether the cause of the invalid value is an attached object or a floating object.

As is clear from the above description, the information integration processing section 80C and the obstacle control section 23H detect obstacles around the work vehicle V based on the detection of the obstacles by the imaging unit 80A and the obstacle detection unit 80B. It functions as a work support unit that notifies the presence of objects and enables work support such as avoiding collisions with obstacles.

As shown in FIG. 7 and FIGS. 15 to 18, the information integration processing unit 80C stores information about the cultivation in the field A based on the color image information from the imaging unit 80A and the measurement information from the first obstacle sensor 86. A growth information acquisition unit 80Ca that acquires growth information of the crop Z that is used as an index for evaluating the growth of the crop Z is included. The growth information acquisition unit 80Ca divides the center side area (work area) A2b of the field A into a plurality of areas to obtain crop Z for each growth information acquisition area (an example of a predetermined area) Ag (see FIGS. 15 to 18). Obtain growth information. The growth information acquisition unit 80Ca obtains, as the growth information of the crop Z, a normalized vegetation index (hereinafter referred to as NDVI), which is an example of a vegetation index indicating the degree of activity of the crop Z, a vegetation cover rate corresponding to the number of stems, and Obtain the plant height value of crop Z, etc.

In addition, the division setting of the growth information acquisition area Ag takes into consideration the front and back length and working width of the work equipment that uses growth evaluation based on the growth information (for example, the fertilizer spreading equipment that spreads fertilizer, etc.). It is preferable to set FIG. 16 is an example of a color image of the front of the vehicle body generated from color image information captured by the front camera 81 of the imaging unit 80A. FIG. 17 is an example of a distance image in front of the vehicle body generated from the measurement information of the first obstacle sensor 86. FIG. 18 is an example of a near-infrared image of the front of the vehicle body generated from the measurement information of the first obstacle sensor 86.

The growth information acquisition unit 80Ca, for example, performs automatic operation of the tractor 1 in an inter-tillage state in which a cultivator (not shown) for inter-tillage work, which is an example of a working device, is connected to the rear of the tractor 1. Growth information acquisition control is executed to acquire growth information of the crop Z for each growth information acquisition area Ag.

The control operation of the growth information acquisition unit 80Ca in the growth information acquisition control will be described below based on the flowchart shown in FIG. 19 and FIGS. 15 to 18.

The growth information acquisition unit 80Ca performs a color image information acquisition process of acquiring color image information of the first imaging range Ri1 imaged by the front camera 81 from the imaging unit 80A, and a growth information acquisition area Ag( A visible red light reflectance acquisition process is performed to acquire the visible red light reflectance R (see FIG. 16) (steps #31 to #32).

The growth information acquisition unit 80Ca performs a measurement information acquisition process of acquiring measurement information by the first obstacle sensor 86 from the obstacle detection unit 80B, and a growth information acquisition area Ag included in the measurement information of the first obstacle sensor 86. Reflection intensity acquisition processing that acquires the reflection intensity of near-infrared laser light from each existing measurement object (see Figure 18), and the average value of each reflection intensity in the acquired growth information acquisition area Ag is calculated as the growth information acquisition area. Near-infrared light reflectance calculation processing is performed to calculate near-infrared light reflectance IR at Ag (steps #33 to #35).

The growth information acquisition unit 80Ca obtains the reflectance R of visible red light and the reflectance of near-infrared light in the same growth information acquisition area Ag acquired by the visible red light reflectance acquisition process and the near-infrared light reflectance calculation process. IR is substituted into the formula for calculating NDVI, NDVI=(IR-R)/(IR+R), and an NDVI calculation process is performed to calculate the NDVI in the growth information acquisition area Ag (step #36).

Thereby, the growth information acquisition section 80Ca can acquire the NDVI based on the color image information from the imaging unit 80A and the measurement information from the first obstacle sensor 86. The value of NDVI takes a value from -1 to 1, and increases as the amount of nitrogen absorbed by crop Z increases and the activity level increases. The growth information acquisition unit 80Ca can generate an NDVI image by coloring, for example, black to white, in accordance with the NDVI value, and can visualize the activity level of the crop Z.

The growth information acquisition unit 80Ca performs a vegetation cover rate acquisition process to acquire the vegetation cover rate in the growth information acquisition area Ag based on the above-mentioned NDVI image (step #37). Then, by taking the product of the NDVI and the vegetation cover rate in the same growth information acquisition area Ag acquired by the NDVI calculation process and the reflection intensity acquisition process, the nitrogen absorption amount of the crop Z in the growth information acquisition area Ag is estimated. A nitrogen absorption amount estimation process is performed (step #38).

The growth information acquisition unit 80Ca includes color image information of the front camera 81 acquired in the color image information acquisition process, the mounting position and mounting angle of the front camera 81 on the tractor 1, and the first obstacle sensor acquired in the measurement information acquisition process. Based on the distance value information included in the measurement information of 86 and the mounting position and mounting angle of the first obstacle sensor 86 on the tractor 1, color image information of the front camera 81 and measurement information of the first obstacle sensor 86 are calculated. A plant height calculation process that acquires the plant height of each crop Z cultivated in the same growth information acquisition area Ag (see Figures 16 to 17) included in the growth information acquisition area A plant height value calculation process is performed to calculate the plant height value of crop Z in Ag (steps #39 to 40).

The growth information acquisition unit 80Ca associates the acquired NDVI, vegetation cover rate, nitrogen absorption amount, plant height value of the crop Z, etc. in the same growth information acquisition area Ag with the position information of the tractor 1 acquired by the positioning unit 30. Then, growth information storage processing is performed to be stored in the on-vehicle storage section 23G (step #41).

In other words, in this work vehicle V, the obstacle detection system 80 provided in the tractor 1 is used to detect obstacles such as NDVI, plant coverage, nitrogen absorption amount, plant height of crop Z, etc. in order to avoid collision with obstacle detection. It can also be used as a growth information acquisition unit that acquires growth information of crop Z grown in field A. As a result, compared to the case where the tractor 1 is equipped with a dedicated growth information acquisition unit, it is possible to acquire the growth information of the crop Z with high accuracy while suppressing the complexity of the configuration of the tractor 1 and the increase in costs. can.

As shown in FIG. 7, the information integration processing unit 80C evaluates the growth status of crops in each growth information acquisition area Ag based on the crop growth information acquired by the growth information acquisition unit 80Ca, and acquires growth information. A growth evaluation section 80Cb that calculates the amount of fertilizer applied for each area Ag is included. The growth evaluation unit 80Cb stores the calculated amount of fertilization for each growth information acquisition area Ag in the in-vehicle storage unit 23G in a state where it is associated with the position information of the tractor 1 acquired by the positioning unit 30.

When the automatic travel control unit 23F automatically travels the tractor 1 to which the fertilizer spreading device, which is an example of a working device, is connected to perform fertilizer application work, the automatic travel control unit 23F stores information in association with the position information of the tractor 1 in the on-vehicle storage unit 23G. The amount of fertilizer applied by the fertilizer spreader is automatically adjusted for each growth information acquisition area Ag based on the amount of fertilizer applied for each growth information acquisition area Ag and the position information of the tractor 1 acquired by the positioning unit 30 during fertilization work. Perform regulatory control.

As a result, it is possible to easily and appropriately improve the variation in crop growth for each growth information acquisition area Ag in field A, and as a result, improve the quality of crops grown in field A and stabilize the yield. be able to.

[Another embodiment]
Another embodiment of the present invention will be described.
Note that the configurations of each of the different embodiments described below are not limited to being applied individually, but can also be applied in combination with the configurations of other different embodiments.

(1) The configuration of the work vehicle V can be modified in various ways.
For example, the work vehicle V may be configured to have a semi-crawler specification traveling vehicle body 1 including left and right crawlers instead of the left and right rear wheels 11.
For example, the work vehicle V may be configured to have a full crawler specification traveling vehicle body 1 including left and right crawlers instead of the left and right front wheels 10 and the left and right rear wheels 11.
For example, the work vehicle V may be configured to have a traveling vehicle body 1 that can only be driven manually.
For example, the work vehicle V may be configured to have an electric specification including an electric motor instead of the engine 14.
For example, the work vehicle V may be configured with a hybrid specification including the engine 14 and an electric motor.

(2) The information integration processing unit 80C and the obstacle control unit 23H, which function as work support units, detect the presence of obstacles around the work vehicle V based on the detection of obstacles only by the obstacle detection unit 80B. The system may be configured to provide work support such as notifying the driver of the problem and avoiding collisions with obstacles.
Further, the obstacle control unit 23H makes it easier to avoid a collision with an obstacle by only performing a notification process to notify that an obstacle exists around the work vehicle V based on the detection of the obstacle. The system may be configured to provide work support such as .

(3) The growth information acquisition unit 80Ca acquires growth information of the crop Z grown in the field A based on the color image information from the rear camera 82 of the imaging unit 80A and the measurement information from the second obstacle sensor 87. The information may be configured to be obtained.

(4) As the third obstacle sensor 88, the right lidar sensor whose measurement range is set to the third measurement range Rm3 to the right from the cabin 13, and the fourth measurement range Rm4 to the left from the cabin 13 is set to the measurement range. The left lidar sensor may also be provided.
In this configuration, the growth information acquisition unit 80Ca determines the growth of the crop Z grown in the field A based on the color image information from the right camera 83 of the imaging unit 80A and the measurement information from the right lidar sensor. It may be configured to obtain information. Further, the growth information acquisition unit 80Ca acquires the growth information of the crop Z grown in the field A based on the color image information from the left camera 84 of the imaging unit 80A and the measurement information from the left lidar sensor. It may be configured as follows.

(5) The growth information acquisition unit 80Ca is configured to acquire, as growth information of the crop Z, a Green Normalized Difference Vegetation Index (GNDVI), which is less affected by the soil between furrows than the NDVI. You can. The green normalized vegetation index calculates the reflectance G of visible green light and the reflectance IR of near-infrared light in the growth information acquisition area Ag using the calculation formula GNDVI=(IR-G)/( It can be calculated by substituting it into IR+G).
Further, the growth information acquisition unit 80Ca is configured to acquire, as growth information of the crop Z, a vertical vegetation index (PVI), a soil adjusted normalized vegetation index (SAVI), etc. You can.

(6) The positioning unit 30 has acquired the growth information such as NDVI, plant cover rate, nitrogen absorption amount, and plant height value of crop Z acquired by the growth information acquisition unit 80Ca, and the evaluation results by the growth evaluation unit 80Cb. You may make it memorize|store in the cloud server in the state linked|related with the positional information of the tractor 1.

(7) The information integration processing section 80C including the growth information acquisition section 80Ca and the growth evaluation section 80Cb may be provided in the on-vehicle control unit 23 or the cloud server.

23E Display control section (work support section)
23F Automatic driving control department (work support department)
23G Storage unit 23H Obstacle control unit (work support unit)
30 Positioning unit 80A Imaging unit 80C Information integration processing section (work support section)
80Ca Growth information acquisition unit 86 First obstacle sensor (rider sensor)
87 Second obstacle sensor (rider sensor)
A Field Ag Growth information acquisition area (designated area)
O Obstacle P Target route Ri1 First imaging range Ri2 Second imaging range Rm1 First measurement range Rm2 Second measurement range V Work vehicle Z Crops

Claims (5)

an imaging unit that is mounted on a work vehicle working in a field and captures visible light in an imaging range set around the work vehicle to obtain color image information;
Mounted on the work vehicle, acquiring measurement information including reflection intensity of near-infrared light by projecting and receiving near-infrared light to a measurement range set around the work vehicle, and based on the acquired measurement information. an obstacle sensor that detects obstacles;
a work support unit that supports work by the work vehicle based on the color image information and the measurement information;
a positioning unit that measures the position of the work vehicle and obtains position information;
a growth information acquisition unit that acquires growth information of crops grown in the field based on the color image information and the measurement information;
A work support system comprising: a storage unit that stores the growth information in association with the position information. The obstacle sensor is a lidar sensor that three-dimensionally measures the distance to the measurement object existing in the measurement range,
The work support system according to claim 1, wherein the growth information acquisition unit acquires a plant height value of the crop as the growth information based on the distance information of the lidar sensor. The growth information acquisition unit acquires the reflectance of visible red light from the color image information, and the reflectance of near-infrared light from the measurement information, and generates a normalized vegetation index as the growth information. The work support system according to claim 1 or 2, wherein the work support system is acquired. The work support unit includes an automatic travel control unit that automatically travels the work vehicle according to a target route generated according to the field based on the position information,
According to any one of claims 1 to 3, the growth information acquisition unit acquires the growth information for each predetermined area set by dividing the field into a plurality of areas as the work vehicle automatically travels. Work support system described. The growth information acquisition unit calculates the amount of fertilization for each predetermined area based on the growth information for each predetermined area,
5. The automatic travel control unit automatically adjusts the amount of fertilizer applied to each predetermined area based on the position information and the amount of fertilizer applied when the work vehicle automatically travels to perform the fertilization work. work support system.

Images