JP7003699B2 - Field work vehicle management system - Google Patents

Description

The present invention relates to a work vehicle management system for working while traveling in a field.

From an air vehicle that can fly over mobile equipment by following mobile equipment such as heavy equipment, an ambient condition detection means such as a camera mounted on the air vehicle, and an ambient condition detection means mounted on the mobile equipment or installed in a remote location. A technique for accurately displaying the surroundings of a mobile device without being affected by vibration of the mobile device or the like by providing a display means for displaying the detection data of the above is known (for example, Patent Document 1).

Japanese Unexamined Patent Publication No. 2016-181119

However, with the above technology, although it is possible to obtain an image of the surroundings of a mobile device from the sky, it is not possible to simultaneously acquire information that cannot be confirmed by a camera image. An object of the present invention is to provide a work vehicle management system capable of simultaneously acquiring information about a vehicle while visually observing the situation around the work vehicle.

The present invention has the following features in order to solve the above problems.

The first invention is equipped with a work vehicle (1) traveling in a field along a planned guide route (R2) by autonomous control and a camera (37), and flies in the air corresponding to the work vehicle (1). The unmanned aerial vehicle (31) is provided with a mobile terminal device (41) that communicates with the work vehicle (1) and the unmanned aerial vehicle (31), and the work vehicle has a vehicle identification information (ID) and a vehicle speed sensor (SN11). ) And the PTO rotation speed sensor (SN14), the mobile terminal device (41) displays the camera image (43a) received from the unmanned aerial vehicle (31) on the display unit (43), and the acquired vehicle. It is characterized in that the vehicle speed information (51) and the PTO rotation information (52) acquired from the work vehicle (1) corresponding to the identification information (ID) are superimposed and displayed on the camera image (43a) .

As a result, since the vehicle information (50) is superimposed and displayed on the image taken from the sky around the vehicle, it is possible to simultaneously acquire the information about the vehicle while visually observing the situation around the work vehicle (1).

Further , since the vehicle information (50) of the work vehicle (1) to be photographed is acquired and displayed on the camera image (43a) displayed on the display unit (43) of the mobile terminal device (41), the work vehicle is displayed. Information about the vehicle can be acquired at the same time while visually observing the situation around (1).

Further , since the vehicle speed information (51) and the PTO rotation speed information (52) of the work vehicle (1) are acquired and displayed on the display unit (43) of the mobile terminal device (41) on the camera image ( 43a ). It is possible to simultaneously acquire information on the vehicle speed and the PTO rotation speed while visually observing the surrounding conditions of the work vehicle (1).

According to the second aspect of the present invention, in the work vehicle management system (S) having the first feature, the work vehicle (1) has a shift sensor (SN13) for detecting a selected shift stage, and the portable terminal device. It is characterized in that the shift stage information (53) is superimposed and displayed on the camera image displayed on the display unit (43) of (41).

As a result, the shift stage information (53) of the work vehicle (1) is acquired and displayed on the display unit (43) of the mobile terminal device (41) at the same time as the camera image (43a). It is possible to simultaneously acquire information on the gear shift of the vehicle while visually observing the situation.

According to the work vehicle management system having the characteristics of the above invention, it is possible to simultaneously acquire information about the vehicle while visually observing the situation around the work vehicle.

It is an overall explanatory view of the work vehicle system of the Example of this invention. It is explanatory drawing when the tractor of FIG. 1 is seen from above. It is explanatory drawing of the functional block diagram which the control system of the work vehicle of an Example of this invention has. It is explanatory drawing of the image displayed on the touch panel of a controller, and is explanatory drawing of the image which the drone captured. It is operation explanatory drawing of the control system of the work vehicle of an Example.

FIG. 1 is an overall explanatory view of a work vehicle system according to an embodiment of the present invention.

Hereinafter, preferred embodiments of the present invention will be described with reference to the drawings.

In each figure, the members unnecessary for the description of the invention are not shown or described as appropriate. Further, in the present specification, the left and right directions facing the forward direction of the work vehicle are referred to as left and right, respectively, the forward direction is referred to as front, and the backward direction is referred to as rear.

In FIG. 1, the work vehicle management system S has an agricultural machine tractor 1 as an example of a work vehicle. The tractor 1 is provided with front wheels 2 and 2 and rear wheels 3 and 3 in the front and rear portions of the airframe, and appropriately decelerates the rotational power of the engine E mounted in the engine room 4 in the front part of the airframe by a transmission device in the transmission case 5. Then, it is configured to transmit to these front wheels 2, 2 and rear wheels 3, 3. The engine room 4 is covered with a bonnet 6. Further, a work machine 200 such as a rotary is mounted on the rear part of the machine body, and the work machine is driven by the PTO shaft.

A cabin 7 is supported on the upper part of the fuselage. Inside the cabin 7, the driver's seat 8 is arranged at the upper position of the transmission case 5, and in front of the driver's seat 8, the steering handle 11, the parking brake (not shown), and the rotation speed of the work equipment are changed. It is configured by arranging a PTO shift lever (not shown) and the like. Further, in front of the driver's seat 8, a speed meter (not shown), various switches for operation (not shown), a communication unit for communication with the outside (not shown), and the like are arranged. A clutch pedal 12 and an accelerator pedal 13 are arranged in the lower front portion of the driver's seat 8.

In FIG. 2, a GPS unit 21 that receives a signal from a positioning satellite and performs positioning is arranged on the upper surface of the roof 7a of the cabin 7 as an example of a vehicle positioning device. The GPS unit 21 is arranged corresponding to the central portion in the vehicle width direction. On the right side of the GPS unit 21, a number mark 22 is attached as an example of the vehicle recognition means. Although the number mark 22 is exemplified as the vehicle identification mark, it is also possible to use it as a graphic mark.

Further, a vehicle position specifying mark 23 is attached to the rear of the GPS unit 21. The vehicle position specifying mark 23 of the embodiment has a rectangular portion 23a and a triangular portion 23b. In the embodiment, the rectangular portion 23a is composed of a black right front portion 23a1, a white left front portion 23a2, a black left rear portion 23a3, and a white right rear portion 23a4, and is configured in a black and white checkered pattern. ing. Further, the triangular portion 23b is attached to the front center of the right front portion 23a1 and the left front portion 23a2, and the front portion is formed in an acute angle shape.

In FIG. 1, the work vehicle control system S has a drone 31 as an example of an unmanned aerial vehicle. The drone 31 of the embodiment has a main body unit 32 provided with a control unit (not shown), a battery (not shown), a communication unit (not shown), and the like. As an example of the positioning device, the main body 32 supports a high-precision GPS unit 33 that receives a signal from a positioning satellite and performs positioning. The main body portion 32 is supported by an arm portion 34 extending from the main body portion toward the outside in the horizontal direction.

A drive source (not shown) is arranged at the tip of the arm portion 34, and a rotary blade 36 is supported by a drive shaft extending in the vertical direction. In the embodiment, the arm portion 34 extends in the cross direction, and each rotary blade 36 is supported at the tip of each arm portion 34. That is, the drone 1 of the embodiment is configured as a so-called quadcopter. A lower camera 37 that images the lower part of the drone 31 as an example of the image pickup member is supported in the lower part of the main body portion 32. Further, a radar 38 for measuring the distance to the lower part is supported in the lower part of the main body portion 32.

The work vehicle control system S has a controller 41 as an example of a mobile terminal device. The controller 41 has a tablet terminal unit 42 as an example of the main body unit. The tablet terminal unit 42 has a control unit (not shown) and a communication unit (not shown). Further, the tablet terminal unit 42 is an example of a display unit, and has a touch panel 43 as an example of an input unit. On the touch panel 43, a camera image or the like of the lower camera 37 of the drone 31 is displayed. Joysticks 44 and 46 as an example of the operation unit are supported on the left and right sides of the tablet terminal unit 42. By operating the left and right joysticks 44 and 46, the drone 31 can be remotely controlled by an operator.

FIG. 3 is an explanatory diagram of a functional block diagram included in the control system of the work vehicle according to the embodiment of the present invention.

In FIG. 3, the work vehicle control system S of the embodiment includes a drone control unit CA, a controller control unit CB, a tractor position information processing control unit CC, and a tractor vehicle control unit CD. Each control unit CA to CD has an input / output interface (I / O) for inputting / outputting signals to and from the outside, a ROM (read-only memory) in which programs and information for performing necessary processing are stored, and necessary. A small information processing device having a RAM (random access memory) for temporarily storing various data, a CPU (central processing unit) that performs processing according to a program stored in a ROM or the like, and an oscillator or the like, so-called It is composed of a microcomputer, and various functions can be realized by executing a program stored in a storage member such as the ROM, RAM, or non-volatile memory.

Here, in the tractor 1 of the embodiment, the position information processing control unit CC and the vehicle control unit CD are configured by a so-called ECU: Electronic Control Unit, and are connected by a CAN: Controller Area Network as a communication line. ing. An external memory Me is connected to the CAN, and the control units CC and CD of the tractor 1 are configured to be accessible to each other to the external memory Me.

In FIG. 3, output signals of a high-precision GPS unit 33, a lower camera 37, a radar 38 for lower distance measurement, a flight state sensor SN1, a communication unit (not shown), and the like are input to the drone control unit CA. ing.

Here, the flight state sensor SN1 of the embodiment is composed of a so-called 6-axis gyro sensor. The flight state sensor SN1 detects the acceleration in each axis direction and the angular velocity around each axis in the three-axis orthogonal directions fixed to the drone 31, and detects the flight state such as the acceleration and the attitude state of the drone 31. ..

In the embodiment, the attitude state of the drone can be detected as the rotation angle of each axis with respect to the direction of gravity by time integration of the angular velocity. That is, it is possible to detect the posture state of the drone 31 by the rotation angle around each axis with respect to the gravity direction (gravitational acceleration direction). In the embodiment, the configuration of the 6-axis gyro sensor is illustrated, but it is also possible to separately provide the sensor for detecting the acceleration and the sensor for detecting the angular velocity. Further, the control unit CA outputs control signals of the motors M1 to M4 of the rotary blade 36, control signals of the communication unit, and the like.

In FIG. 3, the position information processing control unit CC is input with output signals such as a GPS unit 21 and a communication unit (not shown). Further, the position information processing control unit CC outputs a control signal of the communication unit and the like.

In FIG. 3, the position information processing control unit CC has a function of executing processing according to an output signal from the signal output element and outputting a control signal to the control element, which is stored in the controller 41. The field information and the work process information are acquired and stored in the external memory Me.

The vehicle control unit CD inputs vehicle speed information from the vehicle speed sensor SN11, steering angle information of the front wheels 2 and 2 from the steering wheel turning angle sensor SN12, shift stage information from the shift sensor SN13, and PTO rotation information from the PTO rotation speed sensor SN14. .. Further, the vehicle control unit CD is connected to the engine E, the steering motor M11, the transmission HS, the brake cylinder BS, and other control elements (not shown).

In the position information processing control unit CC and the vehicle control unit, the CD communicates information via a wired connection, and the drone control unit CA, controller control unit CB, and vehicle control unit CD are the drone communication unit AU, controller communication unit BU, and vehicle communication unit, respectively. It has a DU and communicates wirelessly with each other. The vehicle control unit CD has a vehicle identification information ID as an example of the vehicle recognition means, and the drone control unit CA and the controller control unit CB can receive the vehicle identification information ID and recognize the vehicle to be photographed.

FIG. 4 is an explanatory diagram of an image displayed on the touch panel of the controller, and is an explanatory diagram of an image captured by the drone. The camera image 43a of the lower camera 37 of the drone 31 is displayed on the touch panel 43. In FIG. 4, in the embodiment, the frame images 43b and 43c are superimposed and displayed on the region corresponding to the lower part of the drone 31 on the camera image 43a. Here, the frame images 43b and 43c are set at positions where the marks 23 and 22 are inside when the drone 31 moves to a preset reference position above the tractor 1.

The control unit CB of the controller 41 processes the camera image 43a of the lower camera 37 to detect the number mark 22 and the vehicle position specifying mark 23. By detecting the number mark 22, the control unit CB of the controller 41 can recognize the vehicle to be photographed.

The control unit CB of the controller 41 performs image processing on the camera image 43a and detects a change in the posture of the tractor 1 based on the shape of the captured mark 23. That is, in FIG. 2, when the front portions 23a1, 23a2 are imaged larger than the rear portions 23a3, 23a4, it can be determined that the tractor 1 is in a posture of climbing an inclined surface. Further, when the rear portions 23a3 and 23a4 are imaged larger than the front portions 23a1,23a2, it can be determined that the tractor 1 is in a posture of descending the inclined surface.

Further, when the right side portions 23a1, 23a4 are imaged larger than the left side portions 23a2, 23a3, it can be determined that the tractor 1 is tilted to the lower left. Further, when the left side portions 23a2 and 23a3 are imaged larger than the right side portions 23a1,23a4, it can be determined that the tractor 1 is tilted to the lower right. That is, the tilted posture of the tractor 1 can be discriminated according to the size of the area between the rectangular portions 23a1 to 23a4 on the camera image 43a.

The control unit CB of the controller 41 displays the captured image 43a of the camera 37 received from the drone control unit CA on the touch panel 43, and the vehicle speed information 51 and PTO rotation received from the vehicle control unit CD of the recognition target vehicle. The vehicle information 50 including the information 52 and the shift stage information 53 is superimposed and displayed on the captured image 43a.

As a result, the touch panel 43 can display the vehicle speed information, the PTO rotation information, and the shift information, which are difficult to grasp only from the image, at the same time as the camera image that can visually confirm the situation around the work vehicle 1, so that the area around the work vehicle can be displayed. It is possible to acquire information about the vehicle at the same time while visually observing the situation.

Further, since the vehicle to be photographed is recognized by the numerical mark 22 of the photographed image 43a or the vehicle identification information ID and the vehicle information is displayed, it is necessary to avoid a problem that the vehicle to be photographed and the vehicle to be displayed in the vehicle information are different from each other. Can be done.

FIG. 5 is an operation explanatory diagram of the control system of the work vehicle of the embodiment. In FIG. 5, in the work vehicle control system S of the embodiment having the above configuration, when the drone 31 is set to the autonomous control mode, the drone 31 rises and the lower camera 37 takes an image of the lower part. The camera image 43a captured by the lower camera 37 is image-processed, and when the marks 22 and 23 of the tractor 1 are detected, the work vehicle is specified based on the detected number marks 22. Further, the direction and the amount of movement of the tractor 1 are calculated based on the detected vehicle position specifying mark 23.

Then, the drone 31 is flight-controlled so as to follow the running of the tractor 1 based on the direction and the amount of movement. Therefore, in the work vehicle control system S of the embodiment, the drone 31 flies corresponding to the position of the tractor 1 by autonomous control.

Here, in the work vehicle control system S of the embodiment, when the tractor 1 moves along the scheduled guide path R2 by autonomous control, the position Tα of the tractor 1 is positioned based on the positioning result of the GPS unit 21. To. For example, even when the operator drives the tractor 1 to the work starting point P3 and moves the tractor 1, the operator refers to the approach guide path R1 and the position Tβ of the tractor 1 and drives the tractor 1 to the working starting point P3 with high accuracy. It is easy to do.

Further, in the embodiment, when it is determined that the drone 31 has reached the reference position, the position Tα of the tractor 1 is positioned by the drone 31. Further, in the embodiment, the image processing used for the autonomous control of the drone 31 is performed not by the control unit CA of the drone 31 but by the control unit CB of the controller 41. Therefore, it is possible to reduce the processing capacity of the control unit CA of the drone 31 as compared with the case of image processing by the control unit CA of the drone 31, and it is easy to simplify the configuration. Therefore, in the embodiment, it is easy to reduce the weight of the drone 31.

Further, the attitude of the drone 31 is controlled so that the vertical direction does not tilt too much with respect to the gravity direction. Therefore, the lower camera 37 is likely to be held in a state of facing downward in the direction of gravity. Therefore, it is possible to detect the posture of the tractor 1 according to the shape of the captured vehicle identification number 23, that is, the area ratio of each portion 23a1 to 23a4 of the rectangular portion 23a.

Further, in the embodiment, the radar 38 provided in the drone 31 measures the height with respect to the tractor 1, and the drone 31 is controlled so that the height between the drone 31 and the tractor 1 becomes constant. Therefore, the positional relationship between the drone 31 and the tractor 1 is likely to be stable.

In the embodiment, a configuration in which the height of the tractor 1 with respect to the cabin 7 is measured by the radar 38 is illustrated, but the present invention is not limited to this. For example, instead of the radar 38, a camera capable of stereo imaging can be provided, and the distance between the tractor 1 and the drone 31 can be measured based on the stereo vision of the tractor 1 and the mark 23. It is also possible to measure the distance between the drone 31 and the tractor 1 based on the focal length in which the vehicle position mark is in focus by the autofocus mechanism of the camera. Instead of the lower camera 37 and the radar 38, it is possible to detect the altitude of the drone 31 based on the altitude sensor.

Further, an external memory Me accessible by each control unit CC and CD is connected to the CAN of the tractor 1 of the embodiment, and the field information, work process information, and the created route R1 are connected to the external memory Me. Information such as R2 is stored. Here, the amount of data such as field information tends to be large. Therefore, in a configuration in which field information or the like is stored in the internal memory of each control unit CC or CD, the amount of data communication flowing through the CAN tends to increase when each control unit CC or CD accesses necessary information. On the other hand, in the embodiment held in the external memory, mutual communication between the control unit CC and the CD tends to be unnecessary, and the load on the CAN can be easily reduced. Therefore, in the CAN of the embodiment, communication delay and the like are less likely to occur.

With the above configuration, the work vehicle management system according to the present embodiment can simultaneously acquire information about the vehicle while visually observing the situation around the work vehicle.

1 Tractor (working vehicle)
22 Number mark (vehicle recognition means)
31 Drone (unmanned aerial vehicle)
37 Camera 41 Controller (mobile terminal device)
42 Tablet terminal 43 Touch panel (display)
43a Camera image 50 Vehicle information 51 Vehicle speed information 52 PTO rotation information 53 Shift stage information ID Vehicle identification information SN11 Vehicle speed sensor SN13 Speed change sensor SN14 PTO rotation speed sensor

Claims (2)

A work vehicle that travels in the field along the planned guidance route by autonomous control ,
An unmanned aerial vehicle equipped with a camera and flying in the air corresponding to the work vehicle,
A mobile terminal device that communicates with the work vehicle and the unmanned aerial vehicle.
The work vehicle has vehicle identification information, a vehicle speed sensor, and a PTO rotation speed sensor.
The mobile terminal device displays a camera image received from the unmanned aerial vehicle on a display unit, and displays vehicle speed information and PTO rotation information acquired from a work vehicle corresponding to the acquired vehicle identification information superimposed on the camera image .
Field work vehicle management system. The work vehicle has a shift sensor that detects the selected shift stage.
The shift stage information is superimposed and displayed on the camera image displayed on the display unit of the mobile terminal device.
The management system for field work vehicles according to claim 1.

Images