JP7006449B2 - Work vehicle management system - Google Patents

Description

The present invention relates to a work vehicle management system for working while traveling in a field.

From an air vehicle that can fly over mobile equipment by following mobile equipment such as heavy equipment, an ambient condition detection means such as a camera mounted on the air vehicle, and an ambient condition detection means mounted on the mobile equipment or installed in a remote location. A technique for accurately displaying the surroundings of a mobile device without being affected by vibration of the mobile device or the like by providing a display means for displaying the detection data of the above is known (for example, Patent Document 1).

Japanese Unexamined Patent Publication No. 2016-181119

However, with the above technology, it is possible to take an image of the surroundings of a mobile device from the sky, but if you want to change the shooting direction, for example, when you want to see the side of the mobile device, you can switch the flying object to manual operation and operate it relative to each other. It was necessary to change the position, and after changing the relative position, it was necessary to control the direction of the camera and shoot the mobile device. It is an object of the present invention to provide a work vehicle management system capable of automatically photographing a work vehicle to be photographed and easily confirming the situation.

The present invention has the following features in order to solve the above problems.

An unmanned aerial vehicle (1) traveling in a field (F), an unmanned aerial vehicle (31) equipped with a camera (61) and flying in the air while communicating with the work vehicle (1), and the work vehicle (1) and the above. The camera (61) includes a terminal device (41) that communicates with the unmanned aerial vehicle (31), the shooting direction changing means (M5, M6) that changes the shooting direction, and the work vehicle (1) is the first. The positioning device (21) is mounted to acquire vehicle position information, the unmanned aerial vehicle (31) is equipped with a second positioning device (33) to acquire unmanned aerial vehicle position information, and the vehicle position information and the unmanned aerial vehicle are obtained. The position information is compared to acquire the front-back relative position (dy) and the up-down relative (dz) position, and among the shooting direction changing means, the camera motor (M6) relating to the pitch angle is the front-back relative position (dy). It is characterized in that the work vehicle (1) is photographed by controlling based on the angle (a2) obtained from the vertical relative position (dz) .

As a result, the direction of the pitch angle of the camera (61) of the unmanned aerial vehicle (31) can be automatically adjusted based on the relative position of the unmanned aerial vehicle (31) and the work vehicle (1), so that the situation around the work vehicle can be easily adjusted. It is possible to confirm to.

According to the second invention, in the work vehicle management system (S) having the first feature, the shooting direction changing means (M5, M6) includes a camera motor (M5) regarding the yaw angle, and the vehicle position information and the said. The lateral relative position (dx) is acquired by comparing the unmanned aerial vehicle position information, and the camera motor regarding the yaw angle is an angle (a1) obtained from the front-rear relative position (dy) and the lateral relative position (dx). The terminal device (41) is controlled based on the vehicle information (50) including the vehicle speed information (51) received from the work vehicle (1), and the camera image (43a) received from the unmanned aerial vehicle (31). Is characterized by overlapping and displaying.

As a result, the vehicle information (50) of the work vehicle (1) is acquired and displayed on the display unit (43) of the terminal device (41) at the same time as the camera image (43a). It is possible to acquire information about the vehicle at the same time while visually observing.

According to the work vehicle management system having the characteristics of the above invention, the situation around the work vehicle can be easily confirmed.

The whole explanatory view of the work vehicle system of the Example of this invention. An explanatory diagram of a functional block diagram included in the control system of the work vehicle according to the embodiment of the present invention. Explanatory diagram of the relative position of the tractor and the drone. Top view of the tractor and drone in the field. Explanatory drawing of the touch panel display image of a controller.

FIG. 1 is an overall explanatory view of a work vehicle system according to an embodiment of the present invention. Hereinafter, preferred embodiments of the present invention will be described with reference to the drawings. In each figure, the members unnecessary for the description of the invention are not shown or described as appropriate. Further, in the present specification, the left and right directions facing the forward direction of the work vehicle are referred to as left and right, respectively, the forward direction is referred to as front, and the backward direction is referred to as rear.

In FIG. 1, the work vehicle management system S has an agricultural machine tractor 1 as an example of a work vehicle. The tractor 1 is provided with front wheels 2 and 2 and rear wheels 3 and 3 in the front and rear portions of the airframe, and appropriately decelerates the rotational power of the engine E mounted in the engine room 4 in the front part of the airframe by a transmission device in the transmission case 5. Then, it is configured to transmit to these front wheels 2, 2 and rear wheels 3, 3. The engine room 4 is covered with a bonnet 6. Further, a work machine 200 such as a rotary is mounted on the rear part of the machine body, and the work machine is driven by a PTO axis (not shown).

A cabin 7 is supported on the upper part of the fuselage. Inside the cabin 7, the driver's seat 8 is arranged at the upper position of the transmission case 5, and in front of the driver's seat 8, the steering handle 11, the parking brake (not shown), and the rotation speed of the work equipment are changed. It is configured by arranging a PTO shift lever (not shown) and the like. Further, in front of the driver's seat 8, a speed meter (not shown), various switches for operation (not shown), a communication unit for communication with the outside (not shown), and the like are arranged. A clutch pedal 12, an accelerator pedal 13, and the like are arranged in the lower front portion of the driver's seat 8.

On the upper surface of the roof 7a of the cabin 7, a GPS unit 21 that receives a signal from a positioning satellite and performs positioning is arranged as an example of a first positioning device that acquires vehicle position information.

The work vehicle control system S has a drone 31 as an example of an unmanned aerial vehicle. The drone 31 of the embodiment has a main body unit 32 provided with a control unit (not shown), a battery (not shown), a communication unit (not shown), and the like. The main body 32 supports a GPS unit 33 that receives a signal from a positioning satellite and performs positioning as an example of a second positioning device that acquires unmanned aerial vehicle position information. The main body portion 32 is supported by an arm portion 34 extending from the main body portion toward the outside in the horizontal direction.

A drive source (not shown) is arranged at the tip of the arm portion 34, and a rotary blade 36 is supported by a drive shaft extending in the vertical direction. In the embodiment, the arm portion 34 extends in the cross direction, and each rotary blade 36 is supported at the tip of each arm portion 34. That is, the drone 1 of the embodiment is configured as a so-called quadcopter.

At the lower part of the main body 32, a first camera motor M5 in which a camera 61 for imaging the lower part of a drone 31 as an example of an imaging member rotates around a vertical axis and a second camera motor M6 rotating around a horizontal axis are used. It is supported by a camera stay 62 having a gimbal mechanism with two degrees of freedom of rotation. Further, a radar 38 as an example of an altitude sensor for measuring a distance H from below is supported in the lower part of the main body portion 32.

The work vehicle control system S has a controller 41 as an example of a terminal device. The controller 41 has a tablet terminal unit 42 as an example of the main body unit. The tablet terminal unit 42 has a control unit (not shown) and a communication unit (not shown). Further, the tablet terminal unit 42 is an example of a display unit, and has a touch panel 43 as an example of an input unit. On the touch panel 43, a camera image or the like of the camera 61 of the drone 31 is displayed. Joysticks 44 and 46 as an example of the operation unit are supported on the left and right sides of the tablet terminal unit 42. By operating the left and right joysticks 44 and 46, the drone 31 can be remotely controlled by an operator.

FIG. 2 is an explanatory diagram of a functional block diagram included in the control system of the work vehicle according to the embodiment of the present invention. In FIG. 2, the work vehicle control system S of the embodiment includes a drone control unit CA, a controller control unit CB, a tractor position information processing control unit CC, and a tractor vehicle control unit CD. Each control unit CA to CD has an input / output interface (I / O) for inputting / outputting signals to and from the outside, a ROM (read-only memory) in which programs and information for performing necessary processing are stored, and necessary. A small information processing device having a RAM (random access memory) for temporarily storing various data, a CPU (central processing unit) that performs processing according to a program stored in a ROM or the like, and an oscillator or the like, so-called It is composed of a microcomputer, and various functions can be realized by executing a program stored in a storage member such as the ROM, RAM, or non-volatile memory.

Here, in the tractor 1 of the embodiment, the position information processing control unit CC and the vehicle control unit CD are configured by a so-called ECU: Electronic Control Unit, and are connected by a CAN: Controller Area Network as a communication line. ing. An external memory Me is connected to the CAN, and the control units CC and CD of the tractor 1 are configured to be accessible to each other to the external memory Me.

In FIG. 2, output signals of a GPS unit 33, a camera 61, a radar 38 for lower distance measurement, a flight state sensor SN1, a communication unit (not shown), and the like are input to the drone control unit CA.

Here, the flight state sensor SN1 of the embodiment is composed of a so-called 6-axis gyro sensor. The flight state sensor SN1 detects the acceleration in each axis direction and the angular velocity around each axis in the three-axis orthogonal directions fixed to the drone 31, and detects the flight state such as the acceleration and the attitude state of the drone 31. ..

In the embodiment, the attitude state of the drone can be detected as the rotation angle of each axis with respect to the direction of gravity by time integration of the angular velocity. That is, it is possible to detect the posture state of the drone 31 by the rotation angle around each axis with respect to the gravity direction (gravitational acceleration direction). In the embodiment, the configuration of the 6-axis gyro sensor is illustrated, but it is also possible to separately provide the sensor for detecting the acceleration and the sensor for detecting the angular velocity. Further, the control unit CA outputs control signals of the motors M1 to M4 of the rotary blade 36, control signals of the communication unit, and the like.

In FIG. 2, the position information processing control unit CC is input with output signals such as a GPS unit 21 and a communication unit (not shown). Further, the position information processing control unit CC outputs a control signal of the communication unit and the like.

In FIG. 2, the position information processing control unit CC has a function of executing processing according to an output signal from the signal output element and outputting a control signal to the control element, and is stored in the controller 41. The field information and the work process information are acquired and stored in the external memory Me.

The vehicle control unit CD inputs vehicle speed information from the vehicle speed sensor SN11, steering angle information of the front wheels 2 and 2 from the steering wheel turning angle sensor SN12, shift stage information from the shift sensor SN13, and PTO rotation information from the PTO rotation speed sensor SN14. .. Further, the vehicle control unit CD is connected to the engine E, the steering motor M11, the transmission HS, the brake cylinder BS, and other control elements (not shown).

The position information processing control unit CC and the vehicle control unit CD communicate information by wired connection, and the drone control unit CA, the controller control unit CB, and the vehicle control unit CD are the drone communication unit AU, the controller communication unit BU, and the vehicle communication unit DU, respectively. And communicate wirelessly with each other. The vehicle control unit CD has a vehicle identification information ID as an example of the vehicle recognition means, and the drone control unit CA and the controller control unit CB can receive the vehicle identification information ID and recognize the vehicle to be photographed.

FIG. 3 is an explanatory diagram of the relative positions of the tractor and the drone. The relative position between the tractor 1 and the drone 31 is calculated from the tractor position information as an example of vehicle position information, the drone position information as an example of unmanned aerial vehicle position information, and the detection value of the radar 38 of the drone 31.

The lateral relative position dx of the drone 31 with respect to the tractor 1 is calculated as a value with the GPS unit 21 of the tractor 1 as the center and the right direction of the tractor 1 as positive. The relative position dy in the front-rear direction is calculated with the rear direction of the tractor 1 as a positive value. The distance H (see FIG. 1) with the ground obtained by the radar 38 of the drone 31 is applied to the vertical relative position dz.

The camera motor M5 regarding the yaw angle of the drone 31 is controlled based on the angle a1 obtained from the ratio of the relative position dy in the front-rear direction and the relative position dx in the lateral direction, and the camera motor M6 regarding the pitch angle is relative to the relative position dz in the vertical direction. The direction of the camera 61 is adjusted so as to always shoot the tractor 1 under the control based on the angle a2 obtained from the position dy.

As a result, the tractor 1 is always displayed on the touch panel 43 of the controller 41 without the need for manual operation, so that the administrator can easily check the surrounding situation of the tractor 1.

FIG. 4 is a plan view of a tractor and a drone traveling in a field. The tractor 1 autonomously operates in the field F along a predetermined planned travel route R. When the flight position of the drone 31 is set by the controller 41, the corresponding lateral relative position dx, the front-back relative position dy, and the vertical relative position dz are selected from the plurality of relative positions stored in advance in the controller control unit CB. The combination of the above is transmitted to the drone control unit CA as a target value, and the drone control unit CA controls the drone 31 so as to match the received target value.

For example, when the rear position P1 of the tractor 1 is specified, the lateral relative position dx is 0, the front-rear relative position dy is a positive predetermined value (for example, 5 m), and the vertical relative position dz is also a positive predetermined value (for example). For example, 7 m) is instructed. In the case of the front position P4, the lateral relative position dx and the vertical relative position dz have the same values as the rear position P1, and only the front-rear relative position dy has a negative predetermined value (for example, −5 m).

When the right side position P2 or the left side position P3 is specified, the front-rear direction relative position dy becomes 0, the lateral direction relative position dx takes a predetermined value, and the vertical direction relative position dz is a position lower than the roof 7a of the cabin 7. (For example, 2 m) is instructed to fly at a position where the tractor 1 can be photographed from the side.

FIG. 5 is an explanatory diagram of a touch panel display image of the controller. The camera image 43a taken by the camera 61 of the drone 31 is displayed on the touch panel 43. The controller control unit CB displays the camera image 43a of the camera 37 received from the drone control unit CA on the touch panel 43, and also displays the vehicle speed information 51, the PTO rotation information 52, and the shift stage information received from the vehicle control unit CD of the vehicle to be photographed. The vehicle information 50 including the 53 is superimposed and displayed on the captured image 43a.

As a result, the touch panel 43 can display the vehicle speed information, the PTO rotation information, and the shift information, which are difficult to grasp only from the image, at the same time as the camera image 43a that can visually confirm the situation around the work vehicle 1. It is possible to acquire information about the vehicle at the same time while visually observing the surrounding conditions.

Further, on the touch panel 43, a drone position button 63 as an example of the flight position setting means is displayed on the photographed image 43a. The currently selected position of the drone position button 63 is displayed in reverse color, and when another button is touch-operated, the horizontal relative position dx, the front-back relative position dy, and the vertical direction corresponding to the operated button are displayed. The combination of relative positions dz is transmitted to the drone control unit CA as a target value.

According to the work vehicle management system according to the above embodiment, it is possible to easily check the situation around the work vehicle from various directions.

1 Tractor (working vehicle)
31 Drone (unmanned aerial vehicle)
41 Controller (terminal device)
43a Camera image 50 Vehicle information 51 Vehicle speed information 61 Camera 63 Drone position button (flight position setting means)
F Field M5 camera motor (means for changing shooting direction)
M6 camera motor (means for changing shooting direction)

Claims (2)

A work vehicle traveling in the field and
An unmanned aerial vehicle equipped with a camera and flying in the air while communicating with the work vehicle,
A terminal device for communicating with the work vehicle and the unmanned aerial vehicle is provided.
The camera has a shooting direction changing means for changing the shooting direction.
The work vehicle is equipped with a first positioning device to acquire vehicle position information and obtain vehicle position information.
The unmanned aerial vehicle is equipped with a second positioning device to acquire unmanned aerial vehicle position information.
The vehicle position information and the unmanned aerial vehicle position information are compared to obtain the front-rear relative position and the vertical relative position.
Among the shooting direction changing means, the camera motor related to the pitch angle is controlled based on the angle obtained from the front-rear direction relative position and the vertical direction relative position, and the work vehicle is photographed.
Work vehicle management system. The shooting direction changing means includes a camera motor related to the yaw angle.
The vehicle position information and the unmanned aerial vehicle position information are compared to obtain the lateral relative position.
The camera motor with respect to the yaw angle is controlled based on the angle obtained from the front-rear relative position and the lateral relative position.
The terminal device superimposes and displays vehicle information including vehicle speed information received from the work vehicle and camera images received from the unmanned aerial vehicle.
The work vehicle management system according to claim 1.

Images