KR20200108949A - Autonomous tractor having crop hight sensing algorithm - Google Patents

Abstract

The present invention relates to an agricultural tractor, and more particularly, to an autonomous tractor using a height sensing algorithm for autonomous driving. The agricultural tractor according to the present invention comprises: a stereo camera installed in front of a tractor; and an autonomous driving control unit for autonomous driving of the tractor. The stereo camera is configured to photograph a front object during autonomous driving of the tractor in a stereo manner. The autonomous driving control unit is configured to calculate the height of the front object on the basis of at least two stereo images input from the stereo camera, and determine whether the autonomous driving of the tractor is to be continuously performed or to be stopped based on the calculated height of the front object.

Description

Autonomous tractor with height sensing algorithm {AUTONOMOUS TRACTOR HAVING CROP HIGHT SENSING ALGORITHM}

The present invention relates to a tractor for agricultural work, and to an autonomous driving tractor using a height sensing algorithm for autonomous driving.

Tractors are indispensable equipment in order to increase the efficiency of farming work per capita in the present age when the agricultural population is decreasing due to the decreasing agricultural population and the aging problem of farmers.

Tractor is a work vehicle that uses traction power to perform various tasks in the agricultural field. Currently, agricultural tractors install a front loader in the front and attach various work attachments to the loader to perform various tasks such as transportation, unloading, and loading. It is common to perform work such as cultivation by mounting a rotor vator or the like through a rear link at the rear part.

A tractor for agricultural work is an equipment that operates on a wide farmland, and is an essential equipment for farmers, but has the disadvantage of performing repetitive work while moving slowly and simply. In addition, farmers are spending a lot of time and energy on a tractor to perform simple and slow work in the agricultural field, where labor is insufficient and work is concentrated in the farming season, which reduces the efficiency of farming work per person and yields There is also a falling problem

In order to solve this problem, an autonomous tractor is known that improves the working efficiency of the tractor by allowing the tractor to move autonomously. In the case of such an autonomous tractor, it is used for various field crops.

In this regard, Korean Patent Publication No. 10-2013-0138463 suggests an autonomous driving system of an agricultural tractor, and the autonomous driving control system of an agricultural tractor according to the present invention includes: a shift actuator for converting a gear of the tractor; An acceleration actuator for adjusting the speed; a wheel encoder for measuring the speed of the tractor; a control panel including a display window indicating the driving mode status, a switch for selecting an autonomous or manual driving mode, and an emergency stop button; autonomous driving of the tractor A data storage unit in which driving information is stored in advance; When the autonomous driving mode is selected in the control panel, the shift actuator by comparing the autonomous driving driving information stored in the data storage unit with a speed value measured and transmitted by the wheel encoder And a control unit that controls the acceleration actuator to correspond to the autonomous driving driving information stored in the data storage unit; and a technology for controlling the speed of the autonomous driving tractor is suggested.

However, when the tractor encounters crops of a certain height or higher in the actual autonomous driving of the tractor, if the tractor continues to run autonomously without avoiding or stopping it, the tractor moves and damages the crops. In this regard, the patent document only suggests a technology for controlling only the autonomous driving speed of a tractor, and for a technology that avoids or stops an obstacle encountered by an autonomous tractor during autonomous driving, especially when it encounters a field crop of a specific height. As it is not suggesting, the efficiency of autonomous driving will decrease.

The present invention is an invention conceived based on the above-described problems, and an object of the present invention is to provide an autonomous tractor with an algorithm for height sensing for autonomous driving to avoid or stop when a tractor encounters a crop of a specific height during autonomous driving. To do.

In order to solve the above-described problem, according to an aspect of the present invention, an autonomous driving tractor having a height sensing algorithm is provided, wherein the tractor includes a stereo camera installed in front of the tractor and a control unit for autonomous driving for autonomous driving of the tractor. And, the stereo camera is configured to photograph an object in front of the tractor in a stereo manner when autonomously driving, and the control unit for autonomous driving is configured to calculate the height of the front object based on at least two stereo images input from the stereo camera. And, based on the calculated height of the front object, it is configured to determine whether to continue the autonomous driving of the tractor or to stop the autonomous driving of the tractor.

In addition, in the above-described aspect, the autonomous driving control unit generates an image receiving unit that acquires images from a plurality of stereo cameras, and a disparity map used to determine the distance and height from the received images. It is configured to further include a corresponding map generating unit for.

In addition, in the above-described aspect, the control unit for autonomous driving is a distance calculator for calculating the distance to the target obstacle or target crop from the corresponding map, and at least the height for calculating the height of the obstacle or crop by using the distance result calculated from the distance calculator. It is configured to further include a calculation unit.

In addition, in the above-described aspect, the stereo camera consists of two cameras, and the two cameras are installed to be spaced apart from each other within a range of 60mm to 70mm at the same height of the tractor.

In addition, in the above-described aspect, the stereo camera includes a first camera and a second camera, and the first camera and the second camera are a charge coupled device (CCD) image sensor and a metal oxide semi-conductor (MOS) image. It may be implemented using any one of sensors such as a sensor, a charge priming device (CPD) image sensor, and a charge injection device (CID) image sensor.

In addition, in the above-described aspect, the correspondence map generator performs matching through the first and second images, and the matching method sets one image among the first image or the second image input as the stereo image as a reference image, A block of a certain area is set around a reference pixel of the selected reference image, and the most similar area corresponding to the block of the certain area of the reference image is found in the remaining unselected images, and a corresponding map is generated by estimating a transition.

In addition, in the above-described aspect, the distance calculator is configured to detect the distance from the camera to the target crop using the corresponding map, then perform edge detection, and perform edge detection to distinguish each of the connected components.

In addition, in the above-described aspect, the height calculation unit is further configured to perform a segmentation process of separating only the target crop in order to measure the height of the target crop among the connected components.

According to the present invention, when a tractor encounters a crop of a specific height during autonomous driving, the height of the crop is calculated, and when the height of the crop is the height that can be driven, the autonomous driving continues while it is determined that the height of the crop is a height that cannot be driven. It is configured to determine the driving so that crops can be prevented from being damaged or damaged by the tractor.In addition, by increasing the mechanization rate of field work, it is possible to drive an autonomous tractor in a field farm, and to secure safety through recognition of obstacles. The effect of being able to contribute can be obtained.

1 is an explanatory diagram for explaining the main configuration of a tractor capable of autonomous driving according to the present invention.
Figure 2 is a block diagram showing the internal configuration of the autonomous driving control unit of the autonomous tractor according to the present invention.
3 is a diagram illustrating an exemplary image processing process in the autonomous driving control unit of the autonomous driving tractor according to the present invention.
Figure 4 is an explanatory diagram for explaining a process of calculating the height of the target crop in the autonomous driving control unit of the autonomous tractor according to the present invention.
5 is a flow chart showing an autonomous driving operation flow of the autonomous driving tractor according to the present invention.
6 is a flow chart showing the main operation of the crop height sensing algorithm used for autonomous driving of the autonomous tractor according to the present invention.

Advantages and features of the present invention, and a method of achieving them will become apparent with reference to embodiments to be described later in detail together with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but will be implemented in various different forms.

In the present specification, the present embodiment is provided to complete the disclosure of the present invention, and to completely inform the scope of the invention to those of ordinary skill in the art to which the present invention belongs. And the invention is only defined by the scope of the claims. Accordingly, in some embodiments, well-known components, well-known operations, and well-known techniques have not been described in detail in order to avoid obscuring interpretation of the present invention.

The same reference numerals refer to the same components throughout the specification. In addition, the terms used in the present specification (referred to) are for describing exemplary embodiments and are not intended to limit the present invention. In this specification, the singular form also includes the plural form unless specifically stated in the phrase. In addition, elements and operations referred to as'comprising (or having)' do not exclude the presence or addition of one or more other elements and operations.

Unless otherwise defined, all terms (including technical and scientific terms) used in the present specification may be used as meanings that can be commonly understood by those of ordinary skill in the art to which the present invention belongs. In addition, terms defined in a commonly used dictionary are not interpreted ideally or excessively unless defined.

Hereinafter, the technical features of the present invention will be described in detail with reference to the accompanying drawings.

1 is a diagram schematically showing an example of a tractor 10 employing a height sensing algorithm for autonomous driving according to the present invention. As shown in Figure 1, a tractor employing a height sensing algorithm (hereinafter referred to as an autonomous driving tractor) 10 includes a plurality of cameras (preferably two stereo cameras) mounted in front of the autonomous driving tractor 10 ( 200) and the autonomous driving control unit 100. The self-driving tractor 10 captures an image of the front of the self-driving tractor running from the two cameras 200, and the autonomous driving control unit 100 is an obstacle positioned in front based on the image input from the camera 200. It is configured to determine the height of (or crop) and transmit a driving command to the engine controller 300 that controls the engine according to the determination result.

2 is a block diagram showing in more detail the configuration of the autonomous driving control unit 100 of the autonomous driving tractor 10 according to the present invention, and FIG. 3 schematically shows the image processing performed by the autonomous driving control unit 100 It is a drawing. 2 and 3, the autonomous driving control unit 100 is configured to receive an image from a plurality of front cameras 200 (C1, C2) and control the operation of the engine control unit 300 based thereon. .

More specifically, the autonomous driving control unit 100 may be implemented with one or more microprocessors operated by a set program, and this set program is included in a height sensing algorithm using a stereo camera according to an embodiment of the present invention to be described later. It can be understood that it contains a series of instructions for performing each step.

The autonomous driving control unit 100 includes an image receiving unit 110 for acquiring images from a plurality of stereo cameras, and a corresponding map generating unit 120 for generating a map used to determine a distance and height from the received images. , Using the distance calculation unit 130 for calculating the distance to the target obstacle or target crop from the camera using the response map 120 and the distance calculation unit 130 It includes a height calculation unit 140 for calculating the height.

The stereo camera 200 protrudes from an upper portion of the front of the tractor and photographs crops in the front path of the tractor in real time. The stereo camera 200 photographs the front of the vehicle through the first camera C1 and the second camera C2. Also, the first camera C1 and the second camera C2 are installed at the same height and have a spacing of 60 mm to 70 mm similar to that of a human eye, and are configured to perform photographing. A first image and a second image are acquired through the first camera C1 and the second camera C2, respectively. The first camera C1 and the second camera C2 include a charge coupled device (CCD) image sensor, a metal oxide semi-conductor (MOS) image sensor, a charge priming device (CPD) image sensor, and a charge coupled device (CID). injection device) may be implemented using any one of sensors such as an image sensor.

Since the stereo camera 200 captures the crop in real time, the driver can grasp the crop without checking whether the crop exists in the tractor path. In addition, the image captured by the stereo camera 200 is transmitted to the image receiving unit 110.

The image receiving unit 110 receives the first image and the second image captured from the stereo camera 200 and stores the same.

The disparity map generator 120 performs matching through the first and second images. The matching method is to set one image among the first image or the second image input as the stereo image as a reference image, and set a block of a certain area around the reference pixel of the selected reference image, This is a stereo matching method for estimating the disparity by finding the most similar area corresponding to the block of the predetermined area of the reference image, and obtaining a correspondence map (also referred to as a disparity map).

The disparity is the x-axis distance difference between the projected pixel coordinates p _l (x _l ,y _l ) and p _r (x _r ,y _r ) of the point P of the image of the left and right stereo camera, and the shooting height of the stereo image is the same. y _l = y _r Therefore, the disparity value of Pl becomes as follows.

Here, xl is the x coordinate of P _l (based on the O _l coordinate system), and xr is the x coordinate of P _r (based on the O _r coordinate system).

The detection of p _r (x _r ,y _r ) matching p _l (x _l ,y _l ) is performed by selecting a pixel that is most similar to pl among y=yl line pixels in the image containing pr. The criteria for determining the similarity may include a sum of squared difference (SSD) that sums the squares of the difference after obtaining the correlation between blocks of a certain area of the left/right image, and a normalized crosscoefficient (NC) using the correlation between pixels. .

Next, the distance calculation unit 130 will be described. As described above, the distance calculation unit is configured to calculate the distance between the stereo camera and the obstacle or crop by using the corresponding map generated by the correspondence map generation unit 120.

The distance calculator 130 may further include removing noise from the above-described correspondence map for accurate distance detection. The distance calculator 130 calculates the distance from the stereo camera to the crop and then performs an edge detection step.

An edge in an image refers to a portion present at a point where the brightness of an image changes from a low value to a high value or vice versa. The edge indicates the boundary of an object in the image, and contains various information such as the ability to detect shape and direction.

The edge detection step is an image processing process showing the boundary of a region in an image. In an embodiment of the present invention, a pixel corresponding to an outline is obtained as a discontinuous point of image brightness. The edge is usually a point where the change in contrast is large based on the contrast of the image. Therefore, it is necessary to detect the rate of change in contrast, brightness, or slope.

As the edge detection method in the present invention, although not limited thereto, edge detection of a first-order differential algorithm using a Sobel, Prewitt, and Roberts mask may be used, and Laplacian, Log (Laplacian of Gaussian), DoG (Difference of Gaussian), or Canny Edge detection, may be used.

Next, the distance calculator 130 connects the edges after detecting the edges, and distinguishes the connected components from each other. Separating the connected components refers to a pre-processing step for extracting the target crop from the background, that is, to classify the overlapping components in the image.

To this end, in the present invention, a certain height range filtering is performed based on the height of the crop center coordinate in the correspondence map, and as a result, a candidate region of the target crop is detected.

As a result, in the present invention, various boundary surfaces including the target crop are detected through edge detection, and pixels included in the crop area are connected to each other by using a connected component method within the detected area. You will be able to tell them apart.

Next, the height calculation unit 140 will be described. After the distance calculator 130 classifies connected components from the image detected by the edge, target object segmentation is performed to measure the height of the target crop. The target crop segmentation refers to a process of extracting only a corresponding crop from the entire picture in order for the height calculation unit 140 to recognize only a portion necessary for estimating the height of the crop.

When the subject for which the height is to be measured is divided from the entire photograph through a series of processes, the height of the crop is calculated using a relational expression using the distance between the camera and the focus and the distance between the camera and the previously calculated subject.

The calculation of the height of the divided target crop will be described below with reference to FIG. 4.

In order to know the actual size of the image formed on the image sensor, it is necessary to know the actual shooting distance (Z) of the captured pixel. The distance Z can be calculated through the difference in the position of the point P photographed on two stereo images, and is as follows.

Where f is the focal length, T is the distance between cameras, x _l is the x coordinate of p _l (based on the Q _l coordinate system), x _r is the x coordinate of p _r (based on the Q _r coordinate system), and x _l -x _r Represents parity (pixel distance formed on left and right images).

Calculate Z coordinate of P(X,Y,Z) → X coordinate calculation possible (see below when the reference coordinate system is the left camera)

Therefore, the height of the target crop in the image can be calculated using the highest pixel of the target crop, and can be calculated as follows.

Crop height = Y + camera height = (Z·(y))/f + camera height

Here, Y denotes the height at which the camera is installed, and y denotes the height of the camera reference coordinate system (pixel height of the image captured by the camera).

Next, with reference to FIG. 5, the operation of the autonomous driving tractor equipped with the crop height sensing algorithm according to the present invention will be described.

5 is a flowchart illustrating an operation flow of an autonomous tractor with a crop height sensing algorithm according to the present invention. As shown in Fig. 3, first, in step S100, the work mode of the self-driving tractor 10 is set by the user. In step S100, a manual driving mode or an autonomous driving mode may be selected by the user.

When the autonomous driving mode is selected in step S110, the autonomous driving control unit 100 proceeds to step S120 to capture an image of crops in front of the driving, and if a manual driving mode other than the autonomous driving mode is selected in step S110, the process proceeds to step S180. End all autonomous driving processes.

In step S120, an obstacle or crop in front is photographed by the stereo cameras C1 and C2, and each of the generated stereo images is received.

In step S130, by using the received stereo image, the autonomous driving controller 100 performs a crop height sensing algorithm for calculating the height of the crop. The crop height sensing algorithm will be described later in more detail with reference to FIG. 6.

When the height of the crop is calculated in step S130, the autonomous driving control unit 100 determines whether driving is possible based on the height of the crop. If it is determined that it can be driven in step S140 (that is, when the height of the crop is less than a predetermined height value), in step S150 the autonomous driving control unit 100 instructs the engine control unit to continue working, and in step S140, it is determined that driving is impossible. If it is determined (that is, when the height of the crop exceeds the predetermined height value), the step proceeds to S160 and the autonomous driving control unit 100 instructs the engine control unit to stop.

In step S170, if the autonomous driving is terminated based on the determination result as in steps S150 and S160, the process proceeds to step S180, and if the autonomous driving is not terminated, the step continues to S110, and steps S110 to S170 are repeated. do.

Next, Figure 6 is a flow chart showing the flow of the crop height sensing algorithm according to the present invention. The flowchart illustrated in FIG. 5 is a diagram corresponding to step S130 in the flowchart of FIG. 3 and shows the operation of step S130 in more detail.

As shown in FIG. 6, an image is captured from a stereo camera in step S120 and a stereo image captured by the image receiving unit 110 of the autonomous driving control unit 100 is respectively received in step S131.

The stereo image received in step S132 is sent to the correspondence map generator 120 and the correspondence map generator 120 generates a correspondence map using the stereo image.

In step S133, the distance calculator calculates the distance to the target crop based on the corresponding map, and in step S134, the edge is detected through edge detection, and the connected components are identified through edge detection, and then classified by components. It is necessary to review whether the separating meaning is correct)

In step S135, a task of dividing only the target crop is performed in order to check the target crop and detect the height of the target crop based on the divided components.

In step S136, the height of the crop for the divided target crop is calculated.

According to the present invention, when a tractor encounters a crop of a specific height during autonomous driving, the height of the crop is calculated, and when the height of the crop is the height that can be driven, the autonomous driving continues, whereas when it is determined that the height of the crop is a height that cannot be driven. It is configured to determine the driving so that crops can be prevented from being damaged or damaged by the tractor.In addition, by increasing the mechanization rate of field work, it is possible to drive an autonomous tractor in a field farm, and to secure safety through recognition of obstacles. The effect of being able to contribute can be obtained.

The above description is merely illustrative of the technical idea of the present invention, and those of ordinary skill in the art to which the present invention pertains will be able to make various modifications and variations without departing from the essential characteristics of the present invention. Accordingly, the embodiments posted in the present invention are for explanation, not for limiting the technical idea of the present invention, and the scope of the technical idea of the present invention is not limited by these embodiments.

Therefore, the scope of protection of the present invention should be interpreted by the following claims rather than limited by the above-described embodiments, and all technical ideas within the scope equivalent thereto should be interpreted as being included in the scope of the present invention.

10: autonomous tractor
100: control unit for autonomous driving
200: stereo camera
110: image receiving unit
120: response map generation unit
130: distance calculator
140: height calculation unit
300: engine control unit

Claims (8)

In an autonomous tractor with a height sensing algorithm,
The agricultural tractor includes a stereo camera installed in front of the tractor and a control unit for autonomous driving for autonomous driving of the tractor,
The stereo camera is configured to photograph an object in front of the tractor in a stereo manner during autonomous driving,
The autonomous driving control unit is configured to calculate the height of the front object based on at least two stereo images input from the stereo camera, and whether to continue the autonomous driving of the tractor based on the calculated height of the front object. Characterized in that it is configured to determine whether to stop autonomous driving
Self-driving tractor with height sensing algorithm.
The method of claim 1,
The control unit for autonomous driving,
The autonomous driving control unit includes an image receiving unit that acquires images from a plurality of stereo cameras, and
Characterized in that it further comprises a correspondence map generator for generating a correspondence map (disparity map) used to determine the distance and height from the received image
Self-driving tractor with height sensing algorithm.
The method of claim 2,
The autonomous driving control unit further includes a distance calculating unit for calculating the distance to the target obstacle or target crop from the corresponding map, and a height calculating unit for calculating the height of the obstacle or crop using at least the distance result calculated from the distance calculating unit. Characterized by
Self-driving tractor with height sensing algorithm.
The method of claim 2,
The stereo camera is composed of two cameras, characterized in that the two cameras are spaced apart from each other within a range of 60mm ~ 70mm at the same height of the tractor.
Self-driving tractor with height sensing algorithm.
The method of claim 1,
The stereo camera is composed of a first camera and a second camera, and the first camera and the second camera are a charge coupled device (CCD) image sensor, a metal oxide semi-conductor (MOS) image sensor, and a charge coupled device (CPD). priming device), characterized in that it is implemented using any one of sensors such as an image sensor and a charge injection device (CID) image sensor.
Self-driving tractor with height sensing algorithm.
The method of claim 2,
The correspondence map generator performs matching through the first and second images, and the matching method sets one image among the first image or the second image input as the stereo image as a reference image, and By setting a block of a certain area around a reference pixel, finding the most similar area corresponding to the block of the certain area of the reference image from the remaining unselected images, and estimating the transition to generate a correspondence map.
Self-driving tractor with height sensing algorithm.
The method of claim 3,
The distance calculator is configured to detect the distance from the camera to the target crop using the corresponding map, then perform edge detection, and perform edge detection to distinguish each of the connected components.
Self-driving tractor with height sensing algorithm.
The method of claim 7,
The height calculator is configured to further perform a segmentation process of separating only the target crop in order to measure the height of the target crop among the connected components.
Self-driving tractor with height sensing algorithm.

Images