JP2828520B2 - Crop row detection device for work equipment - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image pickup means for picking up an image of a field scene of a predetermined range including a plurality of crops arranged in a row in an obliquely forward and downward direction in the body traveling direction, and image information obtained by the image pickup means. A pixel area extracting means for extracting a pixel area corresponding to the crop; a representative point extracting means for extracting a representative point of the pixel area based on the pixel area information extracted by the pixel area extracting means; The present invention relates to a crop line detection device for a working machine, provided with linear approximation means for linearly approximating a line segment connecting representative points extracted by representative point extraction means.

[0002]

2. Description of the Related Art Conventionally, in a crop row detecting apparatus for a work machine of this type, when a representative point of each pixel area is extracted based on pixel area information extracted corresponding to the crop,
The position of the center of gravity of the pixel area is detected and used as a representative point.
Then, the straight line connecting the obtained representative points was determined to be a line segment corresponding to the boundary line between the unplanted side and the already planted side of the crop row.

[0003]

However, in the above-mentioned conventional apparatus, the direction of imaging of the crop by the imaging means is obliquely forward and downward in the body traveling direction. In addition, the difference between the center of gravity position and the root position of the plant differs between the crop located closer to the aircraft and the crop located farther from the aircraft. I couldn't.
Further, when the crop is tilted or falls down, the accuracy of the detection is worse. Therefore, in the method of simply connecting the position of the center of gravity as a representative point, there is a possibility that a line segment corresponding to a crop row cannot be obtained correctly.

SUMMARY OF THE INVENTION The present invention has been made in view of the above circumstances, and an object of the present invention is to reliably identify a planting position of a crop without being influenced by a distance to the crop or a planting state of the crop. It is an object of the present invention to provide a crop line detecting device for a working machine that can accurately detect a boundary between a planting side and a planted side.

[0005]

According to a first characteristic configuration of the crop row detecting apparatus of the present invention, the representative point extracting means is arranged such that, among the pixels constituting the pixel area, the pixels are arranged in a screen coordinate axis direction along a body traveling direction. , A pixel located closest to the base of the crop is set as the representative point.

A second characteristic configuration is that a main axis direction detecting means for detecting a main axis direction of the pixel region is provided.
When the angle between the major axis direction of the main axis direction and the screen coordinate axis along the body advancing direction is out of a predetermined range, the representative point extracting means is configured not to extract the representative point. There is in the point.

[0007] A third characteristic configuration is that, when the representative point extracting means extracts a plurality of pixels located closest to the root of the crop, the representative point extracting means passes through each of the plurality of pixels and passes through the airframe. The point is that the pixel having the larger number of pixels in the pixel area in the direction parallel to the screen coordinate axis along the traveling direction is configured as the representative point.

[0008]

According to the first characteristic configuration, the pixel extracting means is based on image information obtained by imaging the field scene of a predetermined range including a plurality of crops arranged in a row obliquely forward and downward in the machine body traveling direction. In response to the pixel area information extracted corresponding to the crop, the representative point extracting means, among the pixels forming the pixel area, in the direction of the screen coordinate axis along the machine body traveling direction, closest to the root side of the crop. A pixel located is extracted as a representative point of the pixel area, and then a straight line approximation unit draws a straight line connecting these representative points as a line segment corresponding to a boundary between the unplanted side and the already planted side of the crop row. Is approximated by a straight line.

Here, since the pixel located closest to the plant root side of the crop is extracted as the representative point, the planting position of the crop is surely determined by the representative point without being affected by the distance between the aircraft and the crop position. Can be identified as

[0010] According to the second characteristic configuration, the image pickup means captures a pixel image of a predetermined range of field scenes including a plurality of crops arranged in a row in a diagonally forward and downward direction in the machine body traveling direction. With respect to the pixel area information extracted corresponding to the crop by the extraction means, the main axis direction detection means detects the main axis direction of the pixel area, and the screen coordinate axes along the long axis direction and the aircraft traveling direction among the main axis directions. When the angle formed is within a predetermined range, the representative point extracting means is located closest to the root side of the crop in the direction of the screen coordinate axis along the body moving direction among the pixels constituting the pixel area. The pixel is extracted as a representative point.On the other hand, when the angle is out of the predetermined range, the representative point extracting unit does not extract the representative point, and then the linear approximation unit extracts the extracted representative point. Connected straight line, unplanted crop row Linear approximation as a line segment corresponding to the already planted side of the boundary line between.

Here, the major axis direction of the pixel region extracted corresponding to the crop is considered to correspond to the stem direction of the crop, so that the screen coordinate axis along the major axis direction and the machine body traveling direction is used. By judging the angle of the crop, a crop that is greatly inclined or falls can be excluded from the detection target.

[0012] According to the third characteristic configuration, the image pickup means captures the image of the field scene in a predetermined range including a plurality of crops arranged in a row in a diagonally forward and downward direction in the machine body traveling direction, based on the image information. With respect to the pixel area information extracted corresponding to the crop by the extraction means, the representative point extraction means sets the pixels included in the pixel area in the direction of the screen coordinate axis along the machine body traveling direction, on the stock source side of the crop. When a plurality of pixels located closest to each other are extracted, a pixel passing through each of the plurality of pixels and having a larger number of pixels in the pixel region in a direction parallel to a screen coordinate axis along the aircraft traveling direction is defined as: Then, the straight line approximation unit linearly approximates a straight line connecting these representative points as a line segment corresponding to the boundary between the unplanted side and the already planted side of the crop row.

Here, since the direction in which the number of pixels in the pixel region is large is considered to correspond to the direction of the stem of the crop, the pixel located closest to the root side of the pixel region in which the number of pixels in the pixel region is large is considered. By extracting as a representative point, the planting position of the crop can be more accurately identified.

[0014]

Accordingly, the working machine which can reliably identify the planting position of the crop and can accurately detect the boundary between the unplanted side and the already planted side without being affected by the distance to the crop or the planting state thereof. The present invention can provide a crop row detection device for use, and can stably control the steering of a working machine such as a crop planting machine.

[0015]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of a device for detecting the position of a seedling row planted in a field by applying the present invention to a rice planting machine as a crop planting machine will be described below with reference to the drawings.

As shown in FIG. 7 and FIG. 8, a seedling planting device 2 is provided at the rear of an airframe V having both front wheels 1F and rear wheels 1R operable for steering operation. A color image sensor S as an imaging unit for imaging a predetermined range of field scenes including a plurality of planted seedlings T as a plurality of crops planted in a row in the device 2 is provided in front of the machine body V. It is provided on the side.

The mounting structure of the image sensor S will be described. The front end of the support member 4 protruding laterally outward of the body V is provided adjacent to the body V laterally outward of the body V. It is provided so that an image of a seedling row may be taken obliquely from above in the machine traveling direction. That is,
In a state where the body V is properly aligned with a plurality of rows of the already-planted seedlings T arranged in the body traveling direction, a line segment L corresponding to the already-planted seedling row is a line segment L of the imaging field of the image sensor S. The running reference line La passes through the center in the front-rear direction. One image sensor S is provided on each of the right and left sides of the machine body V, and the sensor on the side to be used can be switched left and right.

The control structure of the machine body V will be described. As shown in FIG.
Is transmitted to the front wheel 1F and the rear wheel 1R via the transmission gear 5 so that the speed change operation state by the transmission 5 becomes the operation state corresponding to the preset set traveling speed. Provided, and
The shift electric motor 6 is driven based on the information detected by the shift state detecting potentiometer R3. Further, the front wheel 1F and the rear wheel 1R are configured to be individually power-steered by hydraulic cylinders 7F and 7R, respectively, and a steering angle detecting potentiometer R interlocked with the steering operation of the wheels.
The electromagnetically operated control valves 8F and 8R for operating the hydraulic cylinders 7F and 7R are driven so that the steering angle detected by 1, R2 becomes the target steering angle.

Next, a control configuration for obtaining information that linearly approximates a line segment corresponding to the already planted seedling row based on the imaging information of the image sensor S will be described. As shown in FIG. 1, the image sensor S includes three primary color information R,
G and B are separately output, and the seedling T
By subtracting the blue information B not including the color component of the seedling T from the green information G including the color component
Is extracted. In addition, a subtracter 9 for subtracting the blue information B from the green information G in the form of an analog signal, and outputs the subtractor 9 based on a preset threshold value corresponding to the color of the seedling T A comparator 10 that binarizes the data to output binarized information corresponding to the pixel area Ta, and outputs an output signal of the comparator 10 to a predetermined pixel density (32 ×
An image memory 11 for storing image information corresponding to 32 pixels / 1 screen), and an image memory 1
A microcomputer-based control device 12 is provided for obtaining information for linearly approximating a line segment connecting the pixel regions Ta based on the information stored in 1 and controlling the running based on the information.

In other words, the subtractor 9 and the comparator 10 correspond to the pixel region extracting means 100 for extracting the pixel region Ta corresponding to the seedling T. , The pixel area Ta
Linear approximation means 101 for obtaining information for linear approximation of a line segment L connecting the two, and representative point extracting means 102 for obtaining position information of a point representing the position of the pixel area Ta on the image
Are configured.

Next, the operation of the control device 12 will be described with reference to the flow chart shown in FIG. 2, and the configuration of each part will be described in detail. Each time, the image sensor S performs an imaging process, and thereafter, the pixel region Ta corresponding to the seedling T is extracted by the pixel region extracting means 100 (see FIG. 3). The pixel Ta in FIG. 3 indicates a pixel located at the boundary between the planted portion M and the unplanted portion N of the seedling T.

Here, the operation of the pixel area extracting means 100 will be described in more detail. The intensity of the green component G and the blue component B in the three primary color information output from the image sensor S is determined in the portion where the seedling T exists. Considering the portion corresponding to the mud surface and the water surface portion that reflects natural light, in the portion where the seedlings T are present, the intensity of the green component G is high and the intensity of the blue component B is low. In the portion corresponding to the mud surface, the intensity of both the green component G and the blue component B becomes small. Further, in the water surface portion, the intensity of both the green component G and the blue component B is large. Therefore, the influence of natural light reflected on the water surface and the image information corresponding to the mud surface are removed based on the signal level obtained by subtracting the blue signal B from the green signal G. That is, a portion where the level of the image signal obtained by subtracting the blue signal B from the green signal G is equal to or higher than the set threshold value is extracted and 2
By converting to a value, a pixel corresponding to only the seedling T is extracted.

Next, based on the information stored in the image memory 11, in each of the pixel regions Ta corresponding to the plurality of seedlings T, the seedlings T in the y-axis direction which is the screen coordinate axis along the machine body traveling direction. The pixel located closest to the stock source Tk side (below the imaging screen) is defined as the pixel region T
It is extracted as a representative point P of a (see FIGS. 3A and 3B). That is, the process of obtaining the representative point P in each pixel region Ta corresponds to the representative point extracting means 102. Then, when the representative point extraction processing is completed for all the pixel regions Ta on the screen, a line segment connecting these representative points P is obtained by the linear approximation means 101. Here, a method of obtaining by the Hough transform processing will be described.

The Hough transform will be described. As shown in FIG. 6, the center of the imaging field of view of the image sensor S (x = 1)
6, x = 16) as a reference line in the polar coordinate system, and a plurality of straight lines Ln passing through the respective representative points P are set in advance in a plurality of steps within a range of 0 to 180 degrees with respect to the x axis. The combination of the calculated inclination θ and the distance ρ from the origin, that is, the center of the imaging visual field, is referred to as a linear parameter. ρ = x · cos θ + y · sin θ

Then, for all the representative points P, a two-dimensional histogram for counting the frequency of the linear parameter (ρ, θ) is obtained until the value of the inclination θ set in the plurality of steps reaches 180 degrees. The process of adding is repeated.
When the counting of the frequency of the straight line Ln with respect to all the representative points P is completed, one straight line L is determined from the value added to the two-dimensional histogram in accordance with the combination of the gradient θ and the distance ρ that has the maximum frequency. The straight line L is obtained as linearly approximated information of the line segment L corresponding to the boundary between the unplanted portion N and the already planted portion M of the seedling T on the image screen of the image sensor S.

When the body V is traveling in an appropriate state, the line segment L is a traveling reference line L at the center of the screen.
Since the imaging field of view is set so as to be in a state corresponding to a, the positional deviation amount of the aircraft V in the lateral direction of the aircraft is determined by the positional deviation amount β of the line segment L and the traveling reference line La at the center of the screen. The angle α between the line segment L and the y-axis corresponds to the deviation of the body in the traveling direction (see FIG. 3C). Therefore, the deviation amount β and the angle α
However, by performing the steering control so that both become zero, the aircraft V can be appropriately and automatically driven.

Another Embodiment Next, another embodiment of the operation configuration of the control device 12 will be described with reference to FIG. In this case, the configuration from the image pickup processing by the image sensor S to the extraction operation of the pixel region Ta corresponding to the seedling T is the same as the above-described operation configuration example. Then, the seed T of the seedling T
Before extracting the pixel closest to the k side as the representative point P of each pixel area Ta, the main axis direction of each pixel area Ta is detected by the main axis direction detecting means 103 provided in the control device 12. Then, it is determined whether or not the angle γ between the major axis direction of the main axis direction and the y-axis which is the screen coordinate axis along the body advancing direction falls within a predetermined range (for example, 45 degrees or less). In the pixel area Ta1 out of the range, it is determined that the seedling T is not planted normally, and the representative point P is not extracted (FIG. 5).
(A) (b)).

Then, the representative point extracting means 102 determines the stock of the seedlings T in the y-axis direction, which is the screen coordinate axis along the body moving direction, for all the pixel regions Ta in which the angle γ is within the predetermined range. The pixel located closest to the original Tk side (downward in the imaging screen) is extracted as the representative point P of each of the pixel areas Ta (FIG. 5B).
When the representative point extraction processing is completed, a line segment connecting these representative points P is obtained by the straight line approximation means 101.

Next, another embodiment of the operation configuration of the control device 12 will be described with reference to FIG. In this case, the configuration from the image pickup processing by the image sensor S to the extraction operation of the pixel region Ta corresponding to the seedling T is the same as the above-described operation configuration example. After that, the representative point extracting means 102 extracts a pixel located closest to the seed root Tk side of the seedling T as a representative point P of each pixel region Ta. If the number of pixels located closest to is one, the operation configuration is the same (FIG. 2).
reference). On the other hand, when two or more pixels are extracted,
A pixel passing through the plurality of pixels and having a larger number of pixels in the pixel region Ta in a direction parallel to the screen coordinate axis y-axis along the aircraft traveling direction is set as the representative point P (FIG. 10). (A), (b)). More specifically, based on the drawing, in the three pixel regions Ta in FIG. 10B, the number of pixels closest to the stock source Tk is two in the pixel region Ta2 at the top of the screen. Yes, the pixel area T at the center of the screen
a3 is three, and the pixel area Ta at the bottom of the screen is one. In the pixel area Ta2 in the upper part of the screen, the number of pixels in the direction parallel to the screen coordinate axis y is more likely to pass through the pixel on the left side of the screen, so that the pixel on the left side of the screen is selected as the representative point P. Also, in the pixel area Ta3 at the center of the screen, the number of pixels in the direction parallel to the screen coordinate axis y is likely to pass through the center pixel in the screen left / right direction, so the center pixel in the screen left / right direction is selected as the representative point P. I do. The above processing is unnecessary for the pixel area Ta at the bottom of the screen. Then, when the representative point extraction processing is completed, a line segment connecting these representative points P is obtained by the straight line approximation means 101.

Further, in the above embodiment, the pixel information Ta corresponding to the seedling T is extracted by subtracting the blue information B from the green information G and binarizing it based on the set threshold value. Although illustrated, for example, three primary color information R, G,
Using all of B, a region in which the ratio falls within the set ratio range corresponding to the color of the seedling T may be extracted as the pixel region Ta. The specific configuration of the pixel region extracting means 100 can be variously changed. .

Further, in the above embodiment, the case where the line segment L connecting the representative points P of the respective pixel regions Ta is linearly approximated by using the Hough transform, but the linear approximation is performed by using the least square method or the like. The specific configuration of the linear approximation means 101 can be variously changed.

Further, in the above embodiment, the present invention is applied to a device for automatically running a rice planting machine along a seedling row planted in a field, and information of an approximate straight line corresponding to a detected crop row is applied. Has been exemplified to be used as control information for steering control, but the present invention can be applied to an apparatus for detecting information on approximate straight lines corresponding to various crop rows. The specific configuration of each unit, such as the type of crop and the configuration of the traveling system of the machine, and the use form of the information of the approximate straight line corresponding to the detected crop row can be variously changed.

Incidentally, reference numerals are written in the claims for convenience of comparison with the drawings, but the present invention is not limited to the configuration of the attached drawings by the entry.

[Brief description of the drawings]

FIG. 1 is a block diagram of a control configuration.

FIG. 2 is a flowchart of crop row detection.

FIG. 3 is an explanatory diagram of crop row detection processing on a screen.

FIG. 4 is a flowchart of crop row detection according to another embodiment.

FIG. 5 is an explanatory diagram of crop row detection processing on a screen according to another embodiment.

FIG. 6 is an explanatory diagram of a Hough transform process.

FIG. 7 is a schematic plan view of a crop planting machine.

FIG. 8 is a front view of the same.

FIG. 9 is a flowchart of crop row detection according to another embodiment.

FIG. 10 is an explanatory diagram of crop row detection processing on a screen according to another embodiment.

[Explanation of symbols]

 REFERENCE SIGNS LIST 100 pixel area extracting means 101 straight line approximating means 102 representative point extracting means 103 main axis direction detecting means L line segment P representative point S imaging means T crop Ta pixel area Tk stock origin

──────────────────────────────────────────────────続 き Continued on the front page (58) Fields investigated (Int. Cl. ⁶ , DB name) G06T 1/00 G05D 1/00-1/12 A01C 11/02 331

Claims (3)

(57) [Claims] 1. An image pickup means (S) for picking up an image of a field scene in a predetermined range including a plurality of crops (T) lined up in a row in an obliquely forward and downward direction in a body traveling direction, and an image picked up by the image pickup means (S). A pixel region extracting means (100) for extracting a pixel region (Ta) corresponding to the crop (T) based on the information;
A representative point extracting means (102) for extracting a representative point (P) of the pixel area (Ta) based on the pixel area information extracted by the pixel area extracting means (100), and a representative point extracting means (102). A linear approximation means (101) for linearly approximating a line segment (L) connecting the representative points (P) extracted by the method (1). (102) is a stock origin (Tk) of the crop (T) in a direction of a screen coordinate axis along a body traveling direction among pixels constituting the pixel area (Ta).
A crop line detecting device for a working machine, wherein a pixel located closest to the side is set as the representative point (P). 2. A crop line detecting device for a work machine according to claim 1, further comprising: a main axis direction detecting means (103) for detecting a main axis direction of the pixel area (Ta). When the angle between the long axis direction and the screen coordinate axis along the fuselage advancing direction is out of a predetermined range, the representative point extracting means (102) outputs the representative point (P).
Crop row detecting device for a working machine configured not to extract the crops. 3. The crop row detecting device for a work machine according to claim 1, wherein said representative point extracting means (102) comprises:
When a plurality of pixels located closest to the stock source (Tk) side of the crop (T) are extracted, the pixel passes through each of the plurality of pixels and is parallel to the screen coordinate axis along the aircraft traveling direction. The pixel having the larger number of pixels in the pixel area (Ta) is
A crop row detecting device for a working machine configured to be the representative point (P).