JP3020734B2 - Boundary detection device for autonomous vehicles - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image pickup means for picking up an image of a work place in front of the fuselage advancing direction and a non-work place adjacent thereto in a direction intersecting with the fuselage advancing direction. Area extracting means for extracting an area corresponding to the non-working area based on the information of the area extracted by the area extracting means, and a line corresponding to a boundary between the working area and the non-working area. The present invention relates to a boundary detecting device for an automatic traveling work vehicle provided with a calculating means for calculating the minute.

[0002]

2. Description of the Related Art A boundary detecting device for an automatic traveling work vehicle of this type uses, for example, a rice planting machine as an automatic traveling work vehicle, in which seedlings are planted in units of a plant in a field as a work site adjacent to a ridge as a non-work site. Imaging means for imaging the front side in the body traveling direction in order to obtain control information for automatically traveling along the boundary between the ridge and the field along the body traveling direction, that is, the ridge side, when planting at every set interval in In the captured image information, for example, from the information of the region extracted corresponding to the ridge based on the color information difference between grass occupying most of the ridge and the water surface or mud surface in the field, The calculated line segment is calculated and processed into a straight line or a curve.

Conventionally, in the extracted area, a pixel located closest to the field side on each scanning line on the screen coordinate axis of the imaging means along a direction intersecting with the body moving direction is extracted as a representative point. Then, based on the information of these representative points, a line segment corresponding to the boundary (at the ridge) is obtained.

[0004]

However, in the above-mentioned prior art, when there is, for example, a floating grass having the same color information as the grass on the ridge on the field scene, it cannot be distinguished from the grass on the ridge. However, since the floating grass is always located closer to the field than the grass on the ridge on the scanning line of the screen coordinate axis,
The pixel located closest to the field in the area corresponding to the floating grass or the like is selected as the representative point, and is subjected to the arithmetic processing. Therefore, in this case, a position shifted to the field side from the normal position of the ridge is erroneously detected as the position of the ridge, and as a result, the ridge detection information is used as steering control information. There was a disadvantage that it was difficult.

The present invention has been made in view of the above circumstances, and an object of the present invention is to accurately detect a boundary between a work site such as a ridge and a non-work site without being affected by disturbance factors such as floating grass. It is an object of the present invention to provide a boundary detecting device for an automatic traveling operation.

[0006]

According to the present invention, a boundary detecting device for an automatic traveling work vehicle according to the present invention is characterized in that, in the region extracted by the region extracting means, the boundary detecting device extends along a direction intersecting with a body traveling direction. On each scanning line of the screen coordinate axis of the imaging unit, the number of continuous pixels is a predetermined number or more,
A selection unit that selects only a pixel row closest to the work place side; and a pixel positioned closest to the work place side in each of the pixel rows selected by the selection means is extracted as a representative point. And a calculating unit that calculates the line segment based on the representative points extracted by the representative point extracting unit.

[0007]

According to the characteristic structure of the present invention, based on the difference between color information such as grass occupying most of the ridges as a non-working place and the water surface or muddy surface of a field as a working place, based on the captured image information. Even if there is an area such as floating grass that has the same information as the image information of the non-work area ridge on the field that is the work area, Only a pixel row in which the number of consecutive pixels on each scanning line of the screen coordinate axis along the direction intersecting with the traveling direction is equal to or greater than a predetermined number (for example, a number corresponding to １ ／ of the screen width) is selected and is equal to or less than the predetermined number. Since the pixel rows are not sorted out, the pixel rows corresponding to the above-mentioned floating grass or the like, which usually have a small number of continuous pixels, are removed. Then, a pixel row closest to the work site is selected from among the pixel rows on each scanning line from which error factors such as floating grass have been removed, and further, the pixel row closest to the work site in the selected pixel rows is selected. Are extracted as representative points, and these representative points are subjected to an arithmetic processing for obtaining a line segment corresponding to the boundary between the work place and the non-work place, so that the representative points are not deviated from the original boundary position. The detection accuracy of the line segment corresponding to the boundary can be increased.

[0008]

As described above, the boundary between the working area and the non-working area, such as on a ridge, can be accurately detected without being influenced by disturbance factors such as floating grass. Can be effectively used as steering control information when traveling along the road.

[0009]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, the present invention will be described in which a rice planter as an automatic traveling work vehicle is used to plant seedlings in a field M as a working place adjacent to a ridge N as a non-working place. An example in which the present invention is applied to a device for detecting a ridgeline Nk will be described with reference to the drawings.

As shown in FIGS. 6 and 7, a seedling planting apparatus 2 is provided at the rear of an airframe V having both front wheels 1F and rear wheels 1R operable for steering operation. The plurality of seedlings T are planted by the device 2 in a state of being arranged in six rows. A color-type image sensor S1 is provided in front of the machine body V as an image pickup means for imaging a field M on the front side of the machine body traveling direction and a ridge N adjacent to the field M in a direction intersecting the machine body traveling direction. I have.

The mounting structure of the image sensor S1 will be described. A ridge adjacent to the body V laterally outside the body V is provided at the tip of the support member 4 projecting laterally outside the body V. N and the field M are provided so as to be imaged obliquely from above in the body traveling direction.
That is, the aircraft V is positioned at the ridge Nk along the aircraft traveling direction.
The line segment L corresponding to the ridgeline Nk coincides with the traveling reference line La passing through the center of the imaging field of view of the image sensor S1 in the front-rear direction in a state in which the line Lk is properly aligned. One image sensor S1 is provided on each of the right and left sides of the machine body V, and the sensor on the side to be used can be switched left and right.

To explain the structure of the body V, as shown in FIG. 1, the output of the engine E is transmitted to each of the front wheel 1F and the rear wheel 1R via a transmission 5, and as the operation state of the speed change operation state corresponds to a preset speed, the shifting state detection potentiometer R ₃ provided, and, on the basis of the detection information of the shifting state detecting potentiometer R _3, shift It is configured to drive the electric motor 6 for use.

[0013] Further, the front wheel 1F and the rear wheel 1R are respectively hydraulic cylinders 7F, is configured to be powered steering operation to each other by 7R, potentiometer R ₁ for a steering angle detection interlocked with the steering operation of the wheel, R _The electromagnetically operated control valves 8F and 8R for operating the hydraulic cylinders 7F and 7R are configured to be driven such that the steering angle detected by ₂ becomes the target steering angle.

Therefore, a front steering wheel 1F is a parallel steering system in which the front wheel 1F and the rear wheel 1R are steered in the same phase and at the same angle.
It is possible to selectively use three types of steering types, that is, a four-wheel steering type in which the rear wheels 1R are steered in opposite phases and at the same angle, and a two-wheel steering type in which only the front wheels 1F are changed in direction. However, when the steering operation is automatically performed based on the imaging information of the image sensor S1, the two-wheel steering system is used. When one field trip is completed and the next field trip is performed, the four-wheel steering is used. Steering type and parallel steering type are used. In FIG. 1, S2 is a distance sensor for detecting a traveling distance based on the output rotation speed of the transmission 5.

Next, a control configuration for approximating the line segment L corresponding to the ridgeline Nk based on the image pickup information of the image sensor S1 will be described. As shown in FIG. 1, the image sensor S1 includes three primary color information R, G, B
And binarizing by subtracting blue information B that does not include the color component of grass K from green information G that includes the color component of grass K that occupies most of the ridge N. Thus, a region Ka corresponding to the ridge N is extracted.

In addition, a subtracter 9 for subtracting the blue information B from the green information G in the form of an analog signal,
A comparator 10 that binarizes the output of the subtracter 9 based on a preset threshold value and outputs binarized information corresponding to the area Ka, and outputs an output signal of the comparator 10 to a preset pixel density ( An image memory 11 that stores pixel information corresponding to (32 × 32 pixels / 1 screen), and a line segment L corresponding to the ridgeline Nk based on the information stored in the image memory 11. A control device 1 using a microcomputer that obtains information approximating a straight line or a curve and controls traveling based on the information.
2 are provided.

That is, using the subtractor 9 and the comparator 10, an area extracting means for extracting an area Ka corresponding to the ridge N in which the grass K occupies most, based on image information taken by the image sensor S1. 100 is configured, and in the area Ka extracted by the area extracting means 100 using the control device 12, the screen coordinate axis (y-axis) of the image sensor S1 along the body traveling direction is set. On each scanning line (y = 1 to 32, 32 lines), the number of continuous pixels is equal to or more than a predetermined number (for example, a number corresponding to １ ／ of the screen width, that is, 8), and Means 101 for selecting only the pixel row Kr located closer to the field, and a point which is closest to the field M side in each of the pixel rows Kr selected by the sorting means 101. Calculating means for obtaining a line segment L corresponding to a boundary between the field M and the ridge N based on information on the area Ka extracted by the area extracting means 100; And 102 respectively. And the calculating means 102
Is configured to obtain the line segment L based on the representative point P extracted by the representative point extracting means 103.

Next, the operation of the control device 12 will be described with reference to the flow chart shown in FIG. 2, and the configuration of each part will be described in detail. Each time, the image sensor S1 performs an imaging process, and after this imaging process, the region Ka corresponding to the ridge N is extracted by the region extracting means 100 (FIGS. 3A and 3B). )reference).

Here, the operation of the region extracting means 100 will be described in more detail. The intensity of the green information G and the blue information B of the three primary color information output from the image sensor S1 indicates a portion where the grass K exists, Considering the portion corresponding to the mud and soil surfaces and the water surface portion reflecting natural light, the intensity of the green information G is high and the intensity of the blue information B is low in the portion where the grass K exists. . In the portion corresponding to the mud and the soil surface, the intensity of both the green information G and the blue information B is small. Further, in the water surface portion, the intensity of both the green information G and the blue information B is large. Therefore, the influence of natural light reflected on the water surface and the image information corresponding to the mud and soil surfaces are removed based on the signal level obtained by subtracting the blue information B from the green information G. That is, by extracting and binarizing a portion where the signal level obtained by subtracting the blue information B from the green information G is equal to or higher than the set threshold value, the information of the region Ka corresponding to the grass K can be extracted. In the figure, not only the ridge N side but also the field M
The region Ka corresponding to the grass K is also extracted in FIG.

Next, for each of the 32 horizontal scanning lines on the y-axis of the screen coordinate axis, the selecting process is performed by the selecting means 101. As a result, a predetermined number (for example, 8) of the horizontal scanning lines are obtained. The pixel row Kr located closest to the field M side is selected based on the number of continuous pixels as described above. Therefore, if there is no pixel row Kr having a predetermined number or more of continuous pixels, no pixel row Kr is selected on the horizontal scanning line. Since the number of pixels of the region Ka of floating grass and the like existing on the field M is usually smaller than the predetermined number, the region Ka corresponding to the floating grass and the like is removed by the above-described sorting process. Next, the selected pixel rows Kr
The pixel located closest to the field M side in FIG.
The representative point extraction processing for extracting as
03 (see FIG. 3C). When the selection process and the representative point extraction process are completed for all the horizontal scanning lines on the screen coordinate axis, the line segment L is obtained by the calculating means 102 based on these representative points P. Here, it is configured to be obtained by the Hough transform processing.

To explain the Hough transform, as shown in FIG. 4, the center of the field of view of the image sensor S1 (x =
16, y = 16) as a reference line in the polar coordinate system, a plurality of straight lines passing through the respective representative points P are defined by 0 to 180 degrees with respect to the x axis based on the following equation (i). It is determined as a combination of the inclination θ previously set in a plurality of steps in the range and the distance ρ from the origin, that is, the center of the screen corresponding to the center of the imaging visual field. ρ = y · sinθ + x · cosθ (i)

Then, for all of the representative points P, the frequency of each straight line corresponding to the combination of the parameters (ρ, θ) is counted until the value of the inclination θ set in the plurality of steps reaches 180 degrees. Is repeated for adding the two-dimensional histogram. When the counting of the frequency of the straight line with respect to all the representative points P is completed, the slope θ and the distance ρ, which are the maximum frequencies, are obtained from the value added to the two-dimensional histogram.
Is determined, and one straight line Lx having the maximum frequency is determined, and the straight line Lx is obtained as information obtained by linearly approximating the line segment L corresponding to the ridgeline Nk on the imaging surface of the image sensor S1.

Next, the straight line Lx on the image pickup plane is calculated by using the storage information of the shape and size of the image pickup visual field A of the image sensor S1 on the ground surface measured in advance and the image pickup plane through which the straight line Lx of the maximum frequency passes. Are converted into information on a straight line L on the ground surface based on the pixel positions a, b, c (see FIG. 5). That is, as shown in FIG. 5, a straight line L on the ground surface set as a value of a slope に 対 す る with respect to a running reference line La passing through the center in the width direction of the imaging visual field A in the front-rear direction and a position δ in the width direction. Will be converted into information.

In other words, the lengths l ₀ , l 2 of the two sides at the front and rear positions (y = 0 and y = 32) of the imaging visual field A in the direction intersecting the straight line L corresponding to the ridge Nk ₃₂ , the length l ₁₆ of the imaging field of view A in the width direction at the center of the screen (the pixel position where x = 16, y = 16) and the distance h between the front and rear sides are each actually measured in advance. This will be stored in the control device 12.

Then, a straight line Lx on the imaging plane is
X-coordinate values X ₀ and X _{32 at} pixel positions a and b (positions where y = 0, y = 32) intersecting the x-axis corresponding to two sides at the front and rear positions of the imaging visual field A; The value X ₁₆ of the x coordinate of the position c of the pixel where the straight line Lx intersects the x axis passing through the center of the screen is obtained from the following equation (ii) obtained by modifying the above equation (i). Xi = (ρ−Yi · sin θ) / cos θ (ii) Here, 0, 16, and 32 are substituted for Yi, respectively.

Then, based on the coordinate values on the x-axis obtained by the above equation (ii), the following formulas (iii) and (iv) are used to calculate the position δ in the lateral width direction with respect to the travel reference line La.
And the slope ψ, and the obtained values of the position δ and the slope ψ
The position information is calculated as the position information of the straight line L corresponding to the ridge Nk on the ground surface.

[0027]

(Equation 1)

Therefore, in the steering control for automatically moving the body V along the ridge Nk along the body traveling direction, the inclination of the straight line L with respect to the traveling reference line La
And the position δ in the width direction are both close to zero.
The steering operation will be performed in the form of wheel steering.

The steering control will now be described. The inclination ψ of the straight line L with respect to the travel reference line La, the value of the position δ in the width direction, and the value of the current steering angle φ of the front wheel 1F are described below. , based on (v) below formula, and sets the target steering angle θf of the front wheel 1F, and the current steering angle φ is detected by the steering angle detecting potentiometer R ₁ for the front wheel, the target steering The control valve 8F of the front wheel hydraulic cylinder 7F is driven so as to be maintained within the set dead zone with respect to the direction angle θf. θf = K ₁ δ + K ₂ ψ + K ₃ φ (v) K ₁ , K ₂ , and K ₃ are constants set in advance according to the steering characteristics.

An outline of the turn control for reaching the end of the field stroke and starting toward the beginning of the next field stroke (in this case, on the opposite side of the ridge N) will be described. As the traveling distance detected at the time exceeds the set distance corresponding to the length of one field stroke, the image information of the image sensor S1, for example, the front in the traveling direction with respect to the field M From the position information on the screen of the ridge N adjacent on the side, it is detected that the end of the field process has been reached. When it is determined that the end of the field process has been reached, the planting operation by the seedling planting apparatus 2 is interrupted, the two-wheel steering mode is switched to the four-wheel steering mode, and the maximum cutting time is set for a set time. By maintaining at the corner, the vehicle turns 180 degrees to the next field stroke side. At this time, from the image information of the image sensor S1 on the opposite side to the image sensor S1 used in the previous step, the position of the already planted seedling K in the previous field step adjacent to the next field step is detected. Then, the position in the lateral direction of the aircraft in the field stroke is corrected, and the turn is completed. After the end of the turn, the steering is returned to the two-wheel steering mode, and the steering control in the next field trip is restarted.

[Alternative Embodiment] In the above embodiment, the color image sensor S1 is used as the image pickup means, and the blue information B is subtracted from the green information G and binarized based on the set threshold value. Although the case where K is configured to extract the region Ka corresponding to the ridge N that occupies the majority is illustrated,
For example, using all of the three primary color information R, G, and B, a region whose ratio is within a set ratio range corresponding to the color of the grass K may be extracted as the region Ka. By using a black-and-white image sensor S1 to binarize brightness signals of grass K and other places such as mud surface based on appropriate threshold settings, the area Ka can be extracted by a simple device. Area extraction means 1
The specific configuration of 00 can be variously changed.

In the above embodiment, the screen coordinate axis (y-axis)
On each horizontal scanning line, the selecting means 101 is configured to select a pixel row in which the number of continuous pixels is equal to 1/4 of the screen width, that is, 8 or more. Not limited to eight, for example, grass K on ridge N
If the density of the grass K is dense, set it to 8 or more in order to increase the detection accuracy, while if the density of the grass K is sparse, set it to 8 or less so that detection is not possible. Can be.

Further, in the above embodiment, the case where the line segment L corresponding to the ridgeline Nk is linearly approximated using the Hough transform has been exemplified. The information obtained by approximating a curve can also be obtained, and the specific configuration of the calculation means 102 can be variously changed.

In the above embodiment, the present invention is applied to a rice planter as an automatic traveling work vehicle along a ridge Nk which is a boundary between a field as a work place and a ridge N as a non-work land adjacent thereto. Although the case where the present invention is applied to an apparatus for automatic traveling has been exemplified, the present invention is also applicable to other automatic traveling work vehicles such as lawnmowers, and a work place M and a non-work place N at that time. The configuration of each part, such as the type of the vehicle and the configuration of the traveling system of the aircraft, and the use form of the information of the line segment L corresponding to the detected boundary between the work place M and the non-work place N can be variously changed.

Incidentally, reference numerals are written in the claims for convenience of comparison with the drawings, but the present invention is not limited to the configuration of the attached drawings by the entry.

[Brief description of the drawings]

FIG. 1 is a block diagram of a control configuration.

FIG. 2 is a flowchart of a control operation;

FIG. 3 is an explanatory diagram of image processing.

FIG. 4 is an explanatory diagram of a Hough transform.

FIG. 5 is an explanatory diagram showing a relationship between an aircraft traveling direction and an approximate straight line in an imaging visual field.

FIG. 6 is a schematic plan view of a rice transplanter.

FIG. 7 is a schematic side view of a rice transplanter.

[Explanation of symbols]

 M Work area N Non-work area S1 Imaging means Ka area L line segment Kr Pixel row P Representative point 100 Area extraction means 101 Selection means 102 Operation means 103 Representative point extraction means

──────────────────────────────────────────────────続 き Continuation of the front page (56) References JP-A-2-110605 (JP, A) JP-A-4-112276 (JP, A) JP-A-2-242406 (JP, A) JP-A-2- 301809 (JP, A) JP-A-2-301810 (JP, A) (58) Fields investigated (Int. Cl. ⁷ , DB name) G05D 1/02 A01B 69/00 303 G06F 15/70 330

Claims (1)

(57) [Claims] 1. An image pickup means (S1) for picking up an image of a work place (M) on the front side of the machine body traveling direction and a non-work area (N) adjacent thereto in a direction intersecting the machine body movement direction, and the image pickup means (S1). ), An area extracting means (100) for extracting an area (Ka) corresponding to the non-work place (N), and the area (Ka) extracted by the area extracting means (100). Based on the information of (1), a calculating means (102) for calculating a line segment (L) corresponding to the boundary between the work place (M) and the non-work place (N) is provided. In the detection device, in the area (Ka) extracted by the area extraction means (100), on each scanning line of a screen coordinate axis of the imaging means (S1) along a direction intersecting with a body traveling direction. , The number of consecutive pixels is a predetermined number Or more, and a pixel row positioned displaced in most said working areas (M) side (Kr)
Sorting means (101) for sorting only the pixels, and a pixel located closest to the work place (M) side in each of the pixel columns (Kr) selected by the sorting means (101) is set as a representative point (P). Representative point extracting means for extracting (103)
The calculating means (102) is provided with the representative point extracting means (10).
A boundary detection device for an automatic traveling work vehicle configured to obtain the line segment (L) based on the representative point (P) extracted in 3).