JPH01319878A - Crop line detecting device - Google Patents

Abstract

PURPOSE:To improve the accuracy of an approximate straight line to be obtained and to execute processing at high speed by approximating a segment obtained by connecting specific color areas corresponding to crops based on the positional information of one point to represent one specific color area to a straight line. CONSTITUTION:A representative point extracting means 102 to obtain the positional information of one point in a specific color area Ta to represent the position of the specific color area Ta on a picture is provided. Further, a straight line operation means 101 is composed so as to obtain information to approximate a segment L to the straight line based on the information of the representative point extracting means 102. Thus, the occurrence of an error in the information approximated to the straight line can be prevented, information quantity for the approximation to the straight line can be decreased, and the processing can be executed at high speed.

Description

[Detailed Description of the Invention] [Industrial Application Field] The present invention is based on an imaging means for imaging a predetermined range of a field scene including a plurality of crops arranged in a row, and image information captured by the imaging means. , a specific color area extraction means for extracting a specific color area corresponding to the color of the crop, and a linear operation for obtaining information for linearly approximating a line segment connecting the specific color area based on the information of the specific color area extraction means. The present invention relates to a crop row detection device comprising means for detecting crop rows.

[Conventional technology]

This type of crop row detection device described above is used, for example, when planting seedlings as crops in a field at set intervals in units of plants, such as with a rice transplanter, the machine is aligned in the direction of movement of the machine and the planting is done in the seedling row. In order to use this as control information for automatically traveling along the crop, information is obtained by linearly approximating a line segment connecting specific color areas corresponding to the color of the crop.

By the way, in the past, regardless of the size of the extracted specific color areas, line segments that simply connect the specific color areas were found (see Japanese Patent Application No. 60-2, which the present applicant previously proposed).
(See No. 02688).

[Problem to be solved by the invention]

However, in image processing, captured image information is quantized in two-dimensional directions based on the set pixel density, so the specific color area corresponding to the color of the crop is It is formed by a plurality of pixels.

Therefore, if the line segment that simply connects the specific color area formed by the plurality of pixels is found as in the above conventional method, the position on the image where the straight line corresponding to that line segment passes will be in the specific color area. An error corresponding to the size of will occur.

For example, when performing linear approximation using Hough transform, etc., the number of pixels to be subjected to Hough transform processing, that is, the amount of image information increases, and the calculation time to obtain information to be linearly approximated becomes longer. For example, when the obtained information is used as steering control information for automatically driving a working vehicle along crop rows, there is a disadvantage that control delay becomes large.

The present invention has been made in view of the above-mentioned circumstances, and its purpose is to improve the accuracy of the approximate straight line to be obtained.

[Means to solve the problem]

The crop row detection device according to the present invention includes an imaging means for imaging a field scene in a predetermined range including a plurality of crops arranged in rows, and an identification corresponding to the color of the crop based on image information captured by the imaging means. A specific color region extracting means for extracting a color region, and a straight line calculation means for obtaining information for linearly approximating a line segment connecting the specific color regions based on the information of the specific color region extracting means. The characteristic configuration is that representative point extraction means is provided for obtaining positional information of a point within the specific color area that is representative of the position of the specific color area on the image, and the linear calculation means is configured to extract the representative point. The present invention is configured to obtain information for linearly approximating the line segment based on the information of the means.

[Operation] In other words, for each specific color area extracted, one point within that specific color area is determined as information representing the position of the crop,
Line segments connecting each point representing a specific color area are approximated as a straight line.

Note that the positional information of one point within the specific color area may include a geometric feature point such as the position of the center of gravity on the image of the specific color area,
It is possible to use a specific color area or the like that has been reduced to the size of one pixel by reduction processing in image processing. Further, as the linear calculation means, Hough transform can be used as shown in claim 2.

〔Effect of the invention〕

Therefore, since the line segment connecting the specific color areas is approximated as a straight line based on the positional information of a single point representing the specific color area corresponding to the crop, there may be errors in the linear approximation information depending on the size of the extracted specific color area. can be prevented from occurring, and
Since the amount of information for linear approximation can be reduced, high-speed processing has become possible.

〔Example〕

Hereinafter, an embodiment in which the present invention is applied to a device for detecting the position of a row of seedlings planted in a field by a rice transplanter will be described based on the drawings.

As shown in Figures 7 and 8, a device (2) for planting seedlings is installed at the rear of the fuselage (V), which is configured so that both the front wheels (IF) and the rear wheels (IR) can be freely steered. It is installed so that it can be raised and lowered freely, and the seedlings are planted using the device (2), which is used to plant a plurality of already planted seedlings (as a plurality of crops lined up in rows).
A color image sensor (S I ) serving as an imaging means for capturing an image of a predetermined range of agricultural scenes including T) is provided on the front side of the body (V).

As will be described in detail later, the image sensor (S
Based on the imaging information in l), steering control is performed so that the aircraft (v) automatically travels along the rows of the planted seedlings (T) lined up in the aircraft traveling direction.

In terms of structure, the image sensor (Sl) is attached to the tip of the support member (4) that protrudes laterally outward of the fuselage (V). It is installed so as to image the row of already planted seedlings adjacent to the outside of the plant from diagonally above. In other words, in a state where the aircraft (V) is properly aligned with a row of a plurality of already planted seedlings (T) lined up along the traveling direction of the aircraft, the travel reference line (La) corresponding to the row of already planted seedlings is , the image sensor (
It is arranged so that it passes through the center of the imaging field of view of Sυ in the front-back direction.

Then, a plurality of work strokes from one end of the field to the other end are set in parallel to the width direction of the machine, and in each work stroke, based on the imaging information of the image sensor (S1), the The steering will be controlled so that it automatically travels along the rows of already planted seedlings. However, as will be described in detail later, based on the imaging information of the image sensor (S1),
It is determined whether or not the end of one work stroke has been reached, and as the end is reached, the direction is changed 180 degrees toward the start of the next work stroke adjacent to that work stroke,
It will automatically turn.

Therefore, each time the machine (V) travels in the -stroke, the direction of travel with respect to the field is reversed, and the position of the planted seedlings (T) with respect to the machine (V) is reversed left and right.
Although not described in detail, the image sensors (Sl) are provided on each of the left and right sides of the fuselage (V), and the sensor to be used is switched from left to right for each stroke.

To explain the configuration of the aircraft (V), as shown in FIG. 1, the output of the engine (E) is transmitted to each of the front wheels (IF) and the rear wheels (IR) via a transmission (5). A potentiometer (R1) for detecting a shift state is provided so that the shift operation state of the transmission device (5) corresponds to a preset traveling speed. Botensimeter (R
The transmission electric motor (6) is configured to be driven based on the detected upper part (1).

Further, the front wheels (IF) and the rear wheels (IR) are configured to be operated by power steering separately by hydraulic cylinders (7F) and (7R), respectively, and a steering angle detection system linked to the steering operation of the wheels is configured. The electromagnetically operated control valves (8F) and (8R) that operate the hydraulic cylinders (7F) and (7R) are driven so that the steering angle detected by the potentiometers (R1) and (Rz) becomes the target steering angle. It is configured as follows.

Therefore, there is a parallel steering type in which the front wheels (IF) and the rear wheels (IR) are steered in the same phase and at the same angle, and a type in which the front wheels (IF) and the rear wheels (IR) are steered in opposite phases and at the same angle. 4-wheel steering type, and the front wheels (
It is possible to select and use three types of steering types, including a two-wheel steering type that changes the direction of only the IF).

However, when automatically steering based on the imaging information of the image sensor (S1), the two-wheel steering method is used, and when one work process is completed and the next work process is completed, the above-mentioned two-wheel steering method is used. Four-wheel steering type and parallel steering type are now used.

In FIG. 1, (S2) is a distance sensor for detecting the traveling distance based on the output rotation speed of the transmission (5).

Next, a control configuration for obtaining information for linearly approximating a line segment corresponding to the already planted seedling row based on the imaging information of the image sensor (Sl) will be described.

As shown in FIG. 1, the image sensor (S1) is
It is configured to output the three primary color information (R), (G), and (B) separately, and the green information (G) includes the color component of the seedling (T) but does not include the color component of the seedling (T). By subtracting the blue information (B) and binarizing it, the seedling (
It is configured to extract a specific color area (Ta) corresponding to color T).

To explain, a subtracter (9) subtracts the blue information (El) from the green information (G) in an analog signal state.
, the output of the subtractor (9) is binarized based on a preset darkness value corresponding to the color of the seedling (T) to obtain binarized information corresponding to the specific color area (Ta). A comparator (10) for outputting, an image memory (11) for storing the output signal of the comparator (10) as image information corresponding to a preset pixel density (set to 32×32 pixels/one screen), and Based on the information stored in the image memory (11), information for linearly approximating the line segment connecting the specific color area (Ta) is obtained, and the control device (12) using a microcomputer performs travel control based on the information. ) are provided.

That is, the subtracter (9) and the comparator (10
) extracts a specific color area (Ta) corresponding to the color of the seedling (T) as a crop (100);
and the control device (12)
A line segment (L) connecting the specific color area (Ta) is created using
linear calculation means (101) for obtaining information for linear approximation of
Further, representative point extraction means (102) for obtaining positional information of a point within the specific color area (Ta) representing the position of the specific color area (Ta) on the image are configured.

Next, the configuration of each part will be described in detail while explaining the operation of the control device (12) based on the flowchart shown in FIG.

Every time the aircraft (V) travels a set distance or every set time, the image sensor (Sl) performs imaging processing, and the specific color extraction means (100) corresponds to the seedling (T). A specific color area (Ta) is extracted.

To explain the extraction of the specific color area (Ta), since the color of the seedling (T) is greenish, if we focus on the green information (G) of the three primary color information obtained by statically imaging the field scene, , the value corresponding to the pixel that imaged the seedling (T) is larger than the value of the pixel that imaged other parts such as the mud surface of the field. However, since almost all natural light is reflected on the water surface, the reflected light contains all color components. Therefore, the value of the pre-recorded color information (G) corresponding to the pixel that imaged the water surface is larger than the value of the pixel that imaged other parts, similar to the pixel that imaged the seedling (T).

However, since the seedling (T) is green in color and its muddy surface is brown or gray, the value of the blue component included in its color components is low.

In other words, if we focus on the blue information (B) of the three primary color information captured in the image of the field scene, the value of the pixel that captured the water surface will be large because the reflected light from the water surface contains a golden component, but the value of the pixel that captured the water surface will be large. ) or the pixel that captured the mud surface will have a small value.

Therefore, when the blue information (B) is subtracted from the previously recorded color information (B), only the value of the pixel corresponding to the seedling (T) becomes larger than the value of the pixel that captured the other part, and the subtracted value If it is binarized based on the set darkness value, it is possible to extract image information corresponding only to the seedling (T), that is, information corresponding to the specific color area (Ta) (Fig. 3 (a), ([
1)).

Next, based on the information stored in the image memory (11),
Each of the specific color areas (Ta) corresponding to the plurality of seedlings (T) is labeled, and each of the specific color areas (Ta) is labeled.
) respective centers of gravity (P) (see Figure 3 (0) and (c))
The position on the image is determined as position information of one point representing each specific color area (Ta).

In other words, the process of determining the position of the center of gravity (P) of each specific color area (Ta) on the image is performed by the representative point extraction means (
102).

Then, a line segment connecting the centroids (P) representing each specific area (Ta) found by the representative point extracting means (102) is linearly approximated by Hough transform processing.

However, since the seedlings (T) are planted in multiple rows at the same time in the width direction of the aircraft, the image captured by the image sensor (Sl) shows that the seedlings (T) are set at intervals in the front and back direction and are planted at intervals in the width direction of the aircraft. Also in the direction of planting, a plurality of specific color areas (Ta) corresponding to a plurality of seedlings (T) are arranged in a grid pattern at intervals.

Therefore, although it will not be described in detail, the imaging field of view (eighth ), only the center of gravity (P) of the specific color area (Ta) located furthest to the unpopulated side will be processed as a Hough transform target.

To explain the Hough transform, as shown in FIG. 6, each of the specific regions (T
a) A plurality of straight lines passing through the center of gravity (P) are set at a plurality of inclinations (θ) in advance in a range of 0 to 180 degrees with respect to the X axis based on the following formula (i), Distance from the center of the screen corresponding to the origin or the center of the imaging field of view (ρ)
It is required as a combination of

p = x-srnθ+y-cosθ = −(i)
For each pixel, 1. After repeating the process of adding two-dimensional histograms for counting the frequency of each straight line until the value of the slope (θ) set in the plurality of stages reaches 180 degrees, all the extracted pixels, that is, the The frequency of a plurality of types of straight lines passing through each of the centroids (P) of each specific region is counted for each pixel corresponding to each centroid (P).

When the counting of the frequency of straight lines with respect to the center of gravity (P) of all specific color areas (Ta) is completed, the combination of the slope (θ) and the distance (ρ) that results in the maximum frequency is determined from the values added to the two-dimensional histogram. By determining the maximum frequency, one straight line (Lx) (see Fig. 6) is determined, and that straight line (Lx) is connected to the already planted seedlings (T) on the imaging surface of the image sensor (S1). This information is obtained by approximating the line segment to a straight line.

Next, the straight line viI (Lx) on the image plane is determined based on the stored information of the shape and size of the imaging field of view (A) of the image sensor (S1) on the ground surface measured in advance, and the straight line (Lx) of the maximum frequency. The position of the pixel (a, b,
c) (see FIGS. 4 and 6), the information is converted into information about a straight line (L) on the ground surface.

That is, as shown in FIG. 4, the inclination (ψ) with respect to the steering reference line (L) passing through the center of the width direction of the imaging field of view (8) in the front-rear direction and the position (δ) in the width direction This will be converted into information about a straight line (L) on the ground surface, which is set as a value.

To explain, the imaging field of view (A) is in a direction intersecting a straight line (L) corresponding to a line segment connecting the already planted seedlings (T).
The length of the two front and rear sides of , the distances (h) between the front and rear sides are measured in advance and stored in the control device (12).

Then, the position where the straight line (Lx) on the imaging plane intersects the X axis corresponding to the front and back two sides of the imaging field of view (A) is the pixel position (a, b) (y・16.y・−16). ) and the X coordinate value (Xl6) of the pixel position (c) where the straight line (Lx) intersects the X-axis passing through the center of the screen using the above formula (i). The following (ii
) is obtained from the formula.

COSθ However, Yi is 16 and 0, respectively. -16 will be substituted.

Then, based on the coordinate value on the X axis determined by the above equation (ii), from the following equation (iii) and (iv),
The slope (ψ) with respect to the running reference line (La) and the position (δ) in the width direction are determined, and the calculated slope (ψ) and position (
δ) is calculated as the positional information of the straight line (L) corresponding to the row of planted seedlings on the ground surface.

In other words, the representative point extraction means (1
Information obtained by linearly approximating the line segment connecting the center of gravity (P) of each specific color area (Ta) extracted by 02), that is, the inclination (ψ) and position (δ) in the width direction with respect to the traveling reference line (La). The process of obtaining the above corresponds to the linear calculation means (101).

Therefore, the above-mentioned aircraft (V) is arranged in the direction of movement of the aircraft (
In the steering control for automatically traveling along the row T), two wheels are set so that both the inclination (ψ) with respect to the traveling reference line (La) and the position (δ) in the width direction approach zero.
It will be steered using wheel steering.

In addition, when determining whether or not the end of the work process has been reached based on the imaging information of the image sensor (S1), as shown in FIG. The center of gravity (P) of the specific color area (Ta) corresponding to (T)
) in the aircraft width direction is approximated as a straight line, and the straight line (L) that is the linear approximation passes through the center of the screen at both left and right ends of the imaging field of view (A) in the aircraft width direction. Distance (p1) to the reference line (Lb), (pg
) is obtained as position information corresponding to the distance to the end of the work process.

In other words, since the row of already planted seedlings (T) lined up in the width direction of the machine at the end of the working stroke is detected as a straight line heading in the X-axis direction, the straight line (Lx) that has the maximum frequency in the Hough transform process is The position that passes through both the left and right edges of the screen (
a) and (b). Therefore, the position of the end of the working process is within the imaging field of view (A) on the ground surface.
) can be determined based on the values of the distance 1 (PI) and (h) at both left and right ends with respect to the reference line (Lb) passing through the center in the front-rear direction.

To explain the steering control, the driving reference line (L
Based on the values of the inclination (ψ) and the position (δ) in the width direction with respect to a), and the value of the current steering angle (φ) of the front wheel (IF), the above-mentioned A target steering angle (θf) for the front wheels (IF) is set, and a steering angle detection potentiometer (R3) for the front wheels is set.
control valve (8F) of the front wheel hydraulic cylinder (7F) so that the current steering angle (φ) detected at
will be driven.

1 for θf−, 2 for δ+, 2 for ψ1, φ ・・・・・・(
V) Note that K+, Kg, and Ks are constants that are preset according to the steering characteristics.

If we explain the turn control for reaching the end of a working stroke and turning towards the starting end of the next working stroke,
As the traveling distance detected by the distance sensor (S2) exceeds the set distance set corresponding to the length of one work process, as described above, the distance sensor (S2) passes through the center of the imaging field of view in the width direction. Distance (p) in the front-back direction with respect to the reference line (Lb)
I).

(Pt), and as these values become smaller than preset values, it is determined that the end of the work process has been reached.

When the end of the working process is reached, the seedling planting device (2) interrupts the planting operation and switches from the two-wheel steering type to the four-wheel steering type, and for a set time, By maintaining the maximum cutting angle, the direction is turned 180 degrees toward the next working stroke, and then by switching to the parallel steering type and maintaining the maximum cutting angle for a set time, the next working stroke is performed. The turn will be completed by correcting the aircraft's position in the width direction. After completing the turn, the two-wheel steering mode is restored and steering control is resumed in the next working stroke.

(Separate actual flow side) In the above embodiment, by subtracting the blue information (B) from the green information (G) and binarizing it based on the set darkness value, the specific color area ( For example, the three primary color information (R)
, (G), and (B), the area whose ratio falls within a set ratio range corresponding to the color of the seedling (T) may be extracted as the specific color area (Ta). The specific configuration of the color area extracting means (100) can be modified in various ways.

In addition, in the above embodiment, information for linearly approximating a line segment connecting specific color areas (Ta) extracted using Hough transform is obtained. It is also possible to obtain information approximated by a straight line, and the specific configuration of the straight line calculation means (101) can be changed in various ways.

Furthermore, in the above embodiment, the center of gravity (
Although the case where the position information of P) is used is illustrated, for example, the specific color area (Ta) is reduced to the size of one pixel, and the specific color area (Ta) of the reduced one pixel is Coordinate values on the image may be used as positional information of a single point representing the specific color area (Ta), and the specific configuration of the representative point extracting means (102) can be modified in various ways.

Furthermore, in the above embodiment, the present invention is applied to a device for automatically driving a rice transplanter along rows of seedlings planted in a field, and information on approximate straight lines corresponding to detected rows of crops is used to operate a rice transplanter. Although the case where the present invention is configured to be used as control information for direction control or turn control has been exemplified, the present invention can also be applied to a device for detecting information on approximate straight lines corresponding to various crop rows. The specific configuration of each part, such as the type of crops and the configuration of the traveling system of the aircraft, and the usage form of the information on the approximate straight line corresponding to the detected crop rows can be changed in various ways.

Incidentally, although reference numerals are written in the claims section for convenient comparison with the drawings, the present invention is not limited to the structure shown in the accompanying drawings.

[Brief explanation of the drawing]

The drawings show an embodiment of the crop row detection device according to the present invention, FIG. 1 is a block diagram of the control configuration, FIG. 2 is a flowchart of control operation, and FIGS. 3 (A) to (C) are explanations of image processing. Figure 4 is an explanatory diagram showing the relationship between the imaging field of view on the ground surface and an approximate straight line that goes in the longitudinal direction of the aircraft, and Figure 5 shows the relationship between the imaging field of view at the end of the work stroke and an approximate straight line that goes in the width direction of the aircraft. FIG. 6 is an explanatory diagram of Hough transform, FIG. 7 is a schematic plan view of the rice transplanter, and FIG. 8 is a schematic side view of the same. (Sl)...imaging means, (T)...crops, (Ta)...specific color area, (L)...
...Line segment connecting specific color areas, (100)...Specific color area extraction means, (101) Old...Line line calculation means,
(102)...Representative point extraction means.

Claims (1)

[Claims] 1. An imaging means (S_1) for imaging a predetermined range of field scenes including a plurality of crops (T) arranged in a row;
Based on the image information captured by S_1), the crop (T
); a specific color area extracting means (100) for extracting a specific color area (Ta) corresponding to the color of the color;
100), the crop row detection device is provided with a straight line calculation means (101) for obtaining information for linearly approximating a line segment (L) connecting the specific color area (Ta), based on the information of the above-mentioned Representative point extraction means (102) for obtaining positional information of a point within the specific color area (Ta) representing the position of the specific color area (Ta) on the image is provided, and the linear calculation means (101) A crop row detection device configured to obtain information for linearly approximating the line segment (L) based on information from a representative point extraction means (102). 2. The crop row detection device according to claim 1, wherein the linear calculation means (101) is configured to obtain information for linearly approximating the line segment (L) by Hough transform.