JP2907612B2 - Crop row detector - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION This invention is, intervals apart along the rows
Imaging means for imaging the field plane in a predetermined range including a plurality of crop arranged in Te, based on the captured image information according to the imaging means, and pixel extracting means for extracting pixels corresponding to the crop, the pixel extracting means And a calculating means for calculating a line segment connecting the pixels based on the pixel information extracted by the method.

[0002]

2. Description of the Related Art A crop row detecting apparatus of this kind, when planting seedlings as crops at a set interval in a unit of a plant, such as a rice planting machine, is arranged in the advancing direction of the machine. In order to obtain control information for automatic running along the seedling row, the line connecting the extracted pixels corresponding to the seedling row on the imaging screen is approximated as a straight line or curve, and the existing seedling row with respect to the aircraft position And the control information is obtained.

When the line segment is approximated to a straight line or a curve, for example, when a straight line is approximated using a Hough transform or the like, or when a curve is approximated by another transformation, all the images on the imaged screen are displayed. The information is to be processed. For example, when there are a plurality of seedling rows on the screen, the line segment is obtained by using all pieces of pixel information extracted corresponding to the plurality of seedling rows.

[0004]

However, in the above-mentioned prior art, since all the image information on the imaging screen is subjected to the arithmetic processing, the information of the seedling row other than the row of seedlings that should be detected is processed. Therefore, there is a disadvantage that the detection accuracy when obtaining an approximate line segment corresponding to the original seedling row is reduced due to the mixing of these unnecessary information, and at the same time, the amount of image information to be converted is increased, The calculation time for approximation becomes longer, and when the obtained information is used as steering control information for automatically driving the work vehicle along the crop line, there is a disadvantage that the control delay becomes longer.

The present invention has been made in view of the above circumstances, and an object thereof is to secure the detection accuracy of a desired approximate line segment and at the same time, to improve the processing speed at that time. An object of the present invention is to provide a crop row detecting device.

[0006]

The feature of the crop row detecting apparatus according to the present invention is that, out of the pixel information extracted by the pixel extracting means, the entirety of the screen along the direction of the line segment previously obtained by the arithmetic means is obtained. Direction from the pixel position on the screen of the previously obtained line segment to the direction of the line segment in the range
And a pixel selecting means for selecting only pixels located within a predetermined number of pixels in the above, and the calculating means is configured to set the pixels selected by the pixel selecting means as a calculation target.

[0007]

According to the characteristic structure of the present invention , an interval is provided in the column direction.
In the entire range of the screen along the direction of the previously obtained line segment as the line segment corresponding to the crop line, the calculation is performed from the pixel position on the screen of the previously obtained line segment. Only pixels located within a predetermined number of pixels in a direction intersecting with the direction of the line segment are selected, and since these selected pixels are to be calculated, pixels located in a range greatly deviated from the original crop row position. Is excluded from the target of the calculation, while the pixels located at the original crop row position are reliably selected and subjected to the calculation, and those pixels are connected.
Calculated by Ru line segment corresponding to the current crop row is required.
As a result, the detection accuracy of the approximate line segment obtained by the arithmetic processing can be increased, and at the same time, the number of pixels to be calculated, that is, the amount of information to be processed is reduced, and the calculation time is shortened.

[0008]

Therefore, for example , the pixel of the crop is displayed on the imaging screen.
When there are multiple crop rows that are arranged in rows at intervals
Also, while eliminating pixels other than the original crop row,
The pixels corresponding to the original crop row position can be reliably selected, and the detection accuracy of the approximate line segment required for the crop row can be ensured, and at the same time, the processing speed can be improved. It can be effectively used as steering control information during automatic running of a work vehicle.

[0009]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment in which the present invention is applied to an apparatus for detecting the position of a row of seedlings planted in a field by a rice planting machine will be described below with reference to the drawings.

As shown in FIGS. 7 and 8, a seedling planting device 2 is provided at the rear of an airframe V having both a front wheel 1F and a rear wheel 1R operable for steering operation. The planted seedlings T as a plurality of crops arranged in a row are planted by the device 2. Then, the image sensor S ₁ of the color type of the image pickup means for capturing a farm surface in a predetermined range including those already Uenae T is, the machine body V
Is provided on the front side.

[0011] If described mounting structure of the image sensor S _1, is fixed a base end portion to the seedling planting apparatus 2 located behind the machine body V, the support member 4 stretched toward the fuselage forward fuselage lateral side Image sensor S at the tip of
₁ is attached, and is provided so that an image of an already-planted seedling row adjacent to the machine body V on the lateral outer side of the machine body is obliquely taken from above. That is, in a state where the body V is properly aligned with the row of the plurality of planted seedlings T arranged in the body traveling direction, the line segment L corresponding to the planted seedling Tn adjacent to the unplanted side area is: It is set to be state that matches the running reference line La that passes through the center of the imaging field of view of the image sensor S ₁ in the longitudinal direction. The seedling planting apparatus 2 is mounted on a mud surface of a field via a float 13 so that even if pitching or rolling of the machine body V is performed, the influence of the pitching or rolling is reduced as much as possible. To stabilize the imaging posture.

A plurality of work steps from one end to the other end of the field are set in a state of being arranged in parallel in the machine width direction, and in each work step, based on the imaging information of the image sensor S _1. Thus, the steering is controlled so as to automatically travel along the already planted seedling row. However, as the end of one work stroke is reached, the vehicle is automatically turned in a state of turning 180 degrees toward the start end of the next work stroke adjacent to the work stroke.

Therefore, the running direction of the machine body V with respect to the field is reversed every time the machine body travels one stroke, and the position of the planted seedlings T with respect to the machine body V is in a state of being reversed left and right.
The image sensor S ₁ is the one that provided the people left respectively of the machine body V, resulting in switching the left and right sensors on the side to be used for each stroke. In FIG. 7, the position of the planted seedling T is
Since the right side of the fuselage V, are using the right-hand side of the image sensor S _1.

The structure of the body V will be described. As shown in FIG. 1, the output of the engine E is transmitted to each of the front wheel 1F and the rear wheel 1R via a transmission 5, and the transmission 5 as the operation state of the speed change operation state corresponds to a preset speed, the shifting state detection potentiometer R ₃ provided, and, on the basis of the detection information of the shifting state detecting potentiometer R _3, shift It is configured to drive the electric motor 6 for use.

Further, the front wheel 1F and the rear wheel 1R are configured so that power steering operation is separately performed by hydraulic cylinders 7F and 7R, respectively, and potentiometers R ₁ and R for steering angle detection interlocked with the steering operation of the wheels. _The electromagnetically operated control valves 8F and 8R for operating the hydraulic cylinders 7F and 7R are configured to be driven such that the steering angle detected by ₂ becomes the target steering angle.

Accordingly, a parallel steering system in which the front wheel 1F and the rear wheel 1R are steered in the same phase and at the same angle,
Three types of steering types, that is, a four-wheel steering type in which the front wheel 1F and the rear wheel 1R are steered in opposite phases and at the same angle, and a two-wheel steering type in which only the front wheel 1F is turned, can be selected and used. Has become.

[0017] However, the when automatically steering operation based on the image capturing information of the sensor S _1, together with use of the two-wheel Steering, when completed one working stroke moves to the next work stroke The four-wheel steering type and the parallel steering type are used. In FIG 1, S ₂ is the distance sensor for detecting a travel distance based on the output rotational speed of the transmission 5.

Next, on the basis of the image capturing information of the sensor S _1, the control arrangement will be described for approximating processing line L corresponding to the boundary between the non-plant-side seedling row and SundeUe side seedlings column .

As shown in FIG. 1, the image sensor S
₁ is configured to output the three primary color information R, G, and B separately, and subtracts the blue information B not including the color component of the seedling T from the green information G including the color component of the seedling T to obtain a binary value. Thus, the pixel Ta corresponding to the seedling T is extracted.

In addition, a subtracter 9 for subtracting the blue information B from the green information G in the form of an analog signal,
A comparator 10 for binarizing the output of the subtracter 9 based on a preset threshold value corresponding to the color of the seedling T and outputting binarized information corresponding to the pixel Ta; an output of the comparator 10 An image memory 11 for storing the signal as image information corresponding to a preset pixel density (set to 32 × 32 pixels / one screen), and corresponding to the seedlings T stored in the image memory 11 In the pixel information , the previously obtained line is obtained in the entire range of the screen along the direction of the line segment L ₀ previously obtained by the calculating means 101 described later.
Transportation from the pixel position on the screen of the minute L ₀ in the direction of the line segment L ₀
Only pixels Ta located within a predetermined number of pixels (± d pixels) in the direction of difference are selected (see FIG. 4), and a line segment L connecting these pixels Ta is calculated using these selected pixels Ta as objects of calculation. A microcomputer-based control device 12 that obtains information that approximates a straight line or a curve and that controls travel based on the information is provided.

That is, the subtractor 9 and the comparator 10 determine whether the pixel T corresponding to the color of the seedling T
This corresponds to the pixel extracting means 100 for extracting a. Then, by utilizing the control device 12 and the image memory 11, of the pixels Ta, of the line segment L ₀ previously obtained by the calculating means 101 described later . To cover the entire area of the screen along the direction
Oite, its from a pixel position on the screen of the line segment L ₀ previously obtained
A pixel selecting means 102 for selecting only pixels Ta located within a predetermined number of pixels (± d pixels) in a direction intersecting the direction of the line segment L ₀ , and a line segment L connecting the selected pixels Ta.
Of the calculation means 101 that obtains the values by approximating the values by a straight line or a curve.

Next, the configuration of each part will be described in detail while explaining the operation of the control device 12 based on the flowchart shown in FIG.

Each time the machine V travels a set distance or every set time, an image pickup process is executed by the image sensor S _1, and the pixel Ta corresponding to the seedling T is extracted by the pixel extracting means 100. Will be done.

To explain the extraction of the pixel Ta, since the color of the seedling T is green, focusing on green information G among the three primary color information obtained by imaging the field scene, Is larger than the value of a pixel that has imaged another portion such as a mud surface of a field. However, since natural light is almost totally reflected on the water surface, the reflected light contains all color components. Therefore, the value of the green information G corresponding to the pixel capturing the water surface is larger than the value of the pixel capturing the other portion, similarly to the pixel capturing the seedling T. However, since the seedling T is green, and the mud surface is brown or gray, the value of the blue component contained in the color component is low.

That is, focusing on the blue information B among the three primary color information obtained by imaging the field scene, the reflected light from the water surface contains all the color components, so that the pixel value obtained by imaging the water surface becomes large. The value of the pixel that has captured T or the mud surface is small.

Therefore, when the blue information B is subtracted from the green information G, only the value of the pixel corresponding to the seedling T becomes larger than the value of the pixel obtained by imaging the other portion. When the binarization is performed based on the information, the information of the pixel Ta corresponding to only the seedling T can be extracted (FIG. 4).
(See (a) and (b).) In the figure, eight pixels Ta ₁
状態 Ta ₈ is extracted. However, these 8
Actually, each of the pixels Ta _{1 to} Ta ₈ may be _one pixel in the pixel density (32 × 32 pixels / one screen), but may be one or more pixels.

Next, the pixel selecting means 102
In the information stored in the image memory 11, of the information of the pixel Ta ₁ to Ta ₈ corresponding to the plurality of already Uenae T,
Full range smell of the screen along the direction of the line segment L ₀ previously obtained
Te, the line from the pixel position on the screen of the line segment L ₀ previously obtained
A predetermined number of pixels (± d pixels) in the direction intersecting the direction of minute L ₀
Selecting only pixels Ta ₁ to Ta ₄ is located within (see FIG. 4 (c)). Here, the vertical longitudinal seedlings column direction of the screen, that is, because it is steering control so as to follow the line segment L to be determined, in order to simplify the process, the screen of the line segment L ₀ previously obtained Pixels located within the above-mentioned predetermined number of pixels (± d pixels) are selected from the upper pixel position along both sides in the horizontal direction of the screen (x-axis).

Next, the selected pixels Ta _{1 to} Ta _{4 are selected.}
Is approximated to a straight line or a curve by the calculation means 101. Here, the line segment L is calculated as a straight line by a Hough transform process, which is a straight line approximation converting means.

[0029] If described Hough transform, as shown in FIG. 6, the x axis passing through the center of the imaging field of view of the image sensor S ₁ as a reference line in the polar coordinate system, each pixel Ta
_A plurality of straight lines passing through _{1 to} Ta ₄ are defined by a gradient θ previously set in a plurality of steps in the range of 0 to 180 degrees with respect to the x axis based on the following equation (i), and the origin, that is, the center of the imaging visual field. It is obtained as a combination with the corresponding distance ρ from the center of the screen. ρ = y · sinθ + x · cosθ (i)

After repeating the process of adding a two-dimensional histogram for counting the frequency of each straight line obtained until the value of the inclination θ set in the plurality of steps reaches 180 degrees for one pixel. , The pixels Ta _{1 to} T ₁
The frequency of a plurality of types of straight lines passing through a _4, will be counted for each all the pixels.

When the counting of the frequency of the straight line with respect to each of the pixels Ta _{1 to} Ta ₄ is completed, the combination of the inclination θ and the distance ρ which has the maximum frequency is obtained from the value added to the two-dimensional histogram. One straight line Lx (see FIG. 6) having the maximum frequency is determined, and the straight line Lx is
Will be obtained as the information linear approximation segment L connecting the already Uenae Tn adjacent the non-plant-side region in the imaging plane of the image sensor S _1.

Next, a straight line Lx in the imaging surface, the shape and size of the storage information of the image sensor S ₁ of the imaging field of view A in measuring beforehand the ground surface, the imaging plane a straight line Lx passes the maximum frequency Is converted into information of a straight line L on the ground surface based on the pixel positions a, b, c (see FIGS. 5 and 6). That is, as shown in FIG. 5, a straight line L on the ground surface set as a value of an inclination ψ with respect to a running reference line La passing through the center of the imaging visual field A in the width direction in the front-rear direction and a position δ in the width direction. Will be converted into information.

In addition, the front and rear positions (y = 16 and y = 16) of the imaging visual field A in a direction intersecting a straight line L corresponding to a line segment connecting the planted seedlings Tn adjacent to the unplanted side area.
−16), the lengths l ₁ and l ₃₂ of the two sides and the center of the screen (x = 1)
6, the pixel position at which y = 0), the length l ₁₆ of the imaging field of view A in the horizontal width direction, and the distance h between the front and rear sides.
Are measured in advance and stored in the control device 12.

Then, a straight line Lx on the imaging plane is
X-coordinate values X ₁ and X _{32 at} pixel positions a and b (positions where y = 16, y = −16) intersecting the x-axis corresponding to two sides at the front and rear positions of the imaging visual field A; The x-coordinate value X _{16 of} the image position c where the straight line Lx intersects the x-axis passing through the center of the screen is obtained from the following equation (ii) obtained by modifying the above equation (i). Xi = (ρ−Yi · sin θ) / cos θ (ii) Here, Yi is substituted with 16, 0 and −16, respectively.

Then, based on the coordinate values on the x-axis obtained by the above equation (ii), from the following equations (iii) and (iv), the position δ in the lateral width direction with respect to the traveling reference line La is obtained.
And the slope ψ, and the obtained values of the position δ and the slope ψ
The position information is calculated as the position information of the straight line L corresponding to the already planted seedling row on the ground surface.

[0036]

(Equation 1)

Therefore, in the steering control for automatically moving the machine body V along the rows of the planted seedlings T arranged in the machine body traveling direction, the inclination of the straight line L with respect to the running reference line La and the width の in the width direction are determined. The steering operation is performed in a two-wheel steering system so that both the position δ and the position δ approach zero.

The steering control will now be described. The inclination ψ of the straight line L with respect to the travel reference line La, the position δ in the lateral width direction, and the value of the current steering angle φ of the front wheel 1F are calculated from the following. , based on (v) below formula, and sets the target steering angle θf of the front wheel 1F, and the current steering angle φ is detected by the steering angle detecting potentiometer R ₁ for the front wheel, the target steering The control valve 8F of the front wheel hydraulic cylinder 7F is driven so as to be maintained within the setting dead zone with respect to the direction angle θf. θf = K ₁ δ + K ₂ ψ + K ₃ φ (v) K ₁ , K ₂ , and K ₃ are constants set in advance according to the steering characteristics.

[0039] To describe reaches the end of the working stroke for turning control for turning toward the starting end of the next working stroke, the travel distance detected by said distance sensor S ₂ is one of the working stroke with a to exceed the length the set distance that is set in correspondence, detail but are not, thereby detecting that has reached the working stroke termination from the image information of the imaging unit S _1. When it is determined that the end of the work process has been reached, the planting operation by the seedling planting apparatus 2 is interrupted to switch from the two-wheel steering mode to the four-wheel steering mode, and to cut the maximum cutting time for a set time. By maintaining the angle, the vehicle turns 180 degrees to the next working stroke side, and then switches to the parallel steering type, and maintains the maximum turning angle for a set time.
The position in the lateral direction of the aircraft for the next work process is corrected, and the turn is completed. After the end of the turn, the steering is returned to the two-wheel steering mode, and the steering control in the next work process is restarted.

[0040] In [Other Embodiment] The above embodiment uses an image sensor S ₁ of the color formula as an imaging means for capturing an field surface, 2 value based on a set threshold from the green information G by subtracting the blue information B In the above example, the pixel Ta corresponding to the seedling T is extracted. However, for example, using all of the three primary color information R, G, and B, their ratio corresponds to the color of the seedling T. the region to be a preset ratio range may be extracted as the pixel Ta, or using an image sensor S ₁ for monochrome type as the imaging means, the brightness signal elsewhere, such as seedling locations and mud surface May be binarized based on an appropriate threshold setting to extract pixels with a simple device. The specific configuration of the pixel extracting means 100 can be variously changed.

[0041] Further, in the above embodiment, the support member 4 in which the image sensor S ₁ is attached to the distal end portion is fixed to the seedling planting device 2 located its base end portion to the rear of the machine body V, and the tip portion Although it is configured to extend toward the front of the fuselage on the lateral side of the fuselage, a structure in which the support member 4 is fixed at a position on the front side of the fuselage V may be employed.

In the above embodiment, the pixel selecting means 102
The, for simplification of the process, it is located within a predetermined number of pixels (± d pixels) along both direction of the last calculated line segments L ₀ screen pixel position from the screen horizontal (x-axis direction) was adapted to screen pixels, strictly speaking, since the running direction and the screen horizontal direction not orthogonal, further accurately performed pixel sorting, from a pixel position on the screen of the line segment L ₀ it is also possible to configure so as to screen pixels located within the direction of the predetermined number of pixels (± d pixels) perpendicular to the line segment L _0.

Further, in the above embodiment, the case where the line segment L connecting the pixel Ta corresponding to the crop T is linearly approximated by utilizing the Hough transform has been exemplified. However, for example, the least square method or the like is used. It is also possible to obtain information obtained by linear approximation or curve approximation.
Various changes can be made (see FIG. 3).

Further, in the above embodiment, the present invention is applied to an apparatus for automatically running a rice planting machine along a seedling row planted in a field, and an approximate line segment corresponding to a detected crop row is applied. Although the case where the information is configured to be used as control information for steering control has been described as an example, the present invention can be applied to an apparatus for detecting information on approximate line segments corresponding to various crop rows. The specific configuration of each unit, such as the type of crop and the configuration of the running system of the machine, and the use form of the information on the approximate line corresponding to the detected crop row can be variously changed.

In the claims, reference numerals are provided for convenience of comparison with the drawings, but the present invention is not limited to the configuration of the attached drawings by the entry.

[Brief description of the drawings]

FIG. 1 is a block diagram of a control configuration.

FIG. 2 is a flowchart of a control operation;

FIG. 3 is a flowchart of a control operation according to another embodiment.

FIG. 4 is an explanatory diagram of image processing.

FIG. 5 is an explanatory diagram showing a relationship between an aircraft traveling direction and an approximate straight line in an imaging visual field.

FIG. 6 is an explanatory diagram of the Hough transform.

FIG. 7 is a schematic plan view of a rice transplanter.

FIG. 8 is a schematic side view of the same.

[Explanation of symbols]

L segment L ₀ segment S ₁ imaging means T crops Ta pixel Ta ₁ to Ta ₄ pixel 100 pixel extracting means 101 computing means 102 pixel selection means

────────────────────────────────────────────────── ─── Continuation of the front page (56) References JP-A-3-19606 (JP, A) JP-A-4-293109 (JP, A) JP-A-64-74680 (JP, A) (58) Investigation Field (Int.Cl. ⁶ , DB name) A01B 69/00 303 A01C 11/02 331 G05D 1/02 G06T 1/00 G06T 7/00

Claims (1)

(57) [Claims] 1. A with imaging means for imaging the field plane in a predetermined range including a plurality of crop arranged at intervals along the rows (T) (S _1), an image captured by the imaging means (S ₁₎ Pixel extracting means (100) for extracting a pixel (Ta) corresponding to the crop (T) based on the information; and the pixel (Ta) based on the pixel information extracted by the pixel extracting means (100). Calculating means (1) for obtaining a line segment (L) connecting
01) is provided, wherein the pixel information extracted by the pixel extracting means (100) in the direction of the line segment (L ₀ ) previously obtained by the calculating means (101). Pixels (Ta _{1 to} Ta) located within a predetermined number of pixels in a direction intersecting the direction of the line segment (L ₀ ) from the pixel position on the screen of the previously obtained line segment (L ₀ ) in the entire range of the screen along the line. ₄ ) A pixel selecting means (102) for selecting only the above is provided, and the calculating means (101) comprises:
A crop row detecting device configured to target the pixels (Ta _{1 to} Ta ₄ ) selected by the pixel selecting means (102) for calculation.