JPH05165518A - Crop string detector - Google Patents

Abstract

PURPOSE:To secure the detection accuracy of a derived approximate line and also, to improve the processing speed by setting only a selected picture element as an object of an arithmetic operation. CONSTITUTION:By a selective means 102, only picture elements Ta1-Ta4 positioned in less than the number of prescribed picture element (+ or - d picture elements) from a pixel position on the screen of a segment L0 derived in the previous time are selected, in information picture elements Ta1-Ta8 corresponding to plural pieces of already planted seedlings, in storage information of an image memory. Subsequently, the segment for connecting the selected picture elements Ta1-Ta4 is subjected to approximate calculation to a straight line or a curve by an arithmetic means 101. In such a manner, the detection accuracy of a derived approximate segment can be secured and also, the processing speed in that case can be increased.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image pickup means for picking up an image of a field scene in a predetermined range including a plurality of crops arranged in a row, and a pixel corresponding to the crop based on imaged image information by the image pickup means. The present invention relates to a crop row detecting device provided with a pixel extracting means for extracting a line segment and an arithmetic means for obtaining a line segment connecting the pixels based on the pixel information extracted by the pixel extracting means.

[0002]

2. Description of the Related Art A crop row detecting device of the above type has a structure in which, when a seedling as a crop is planted in a field in set intervals at a set unit, such as a rice planting machine, the machines are aligned in the machine traveling direction. In order to obtain control information for automatic traveling along the seedling row, the line segment connecting the pixels extracted corresponding to the seedling row on the imaging screen is approximated to a straight line or a curve and the seedling row that has already been planted for the machine position The control information is obtained by detecting the position and direction of the.

When the above-mentioned line segment is approximated to a straight line or a curved line, for example, when the Hough transform is used to perform the linear approximation, or when the curve is approximated by another conversion, all the images on the imaging screen are displayed. Information is processed, and for example, when there are a plurality of seedling rows on the screen, all the pixel information extracted corresponding to the plurality of seedling rows is used to obtain the line segment.

[0004]

However, in the above-mentioned prior art, since all image information on the image pickup screen is subject to arithmetic processing, information on seedlings other than the seedlings to be originally detected is also processed. Therefore, there is a disadvantage in that the detection accuracy when obtaining the approximate line segment corresponding to the original seedling is reduced due to the mixing of these unnecessary information, and at the same time, the amount of image information to be converted becomes large and the line information increases. There is also a disadvantage that the calculation time for the minute approximation becomes long and the control delay becomes large when the obtained information is used as steering control information for automatically traveling the work vehicle along the crop row.

The present invention has been made in view of the above circumstances, and an object thereof is to secure the detection accuracy of an approximate line segment to be obtained, and at the same time, to improve the processing speed at that time. An object is to provide a crop row detection device.

[0006]

The feature structure of the crop row detecting device according to the present invention is that, from among the pixel information extracted by the pixel extracting means, from the pixel position on the screen of the line segment previously obtained by the calculating means, Pixel selection means for selecting only pixels located within a predetermined number of pixels is provided, and the calculation means is configured to target the pixels selected by the pixel selection means.

[0007]

According to the characteristic configuration of the present invention, only the pixels located within a predetermined number of pixels from the pixel position on the screen of the line segment obtained last time as the line segment corresponding to the crop row are selected, and these selected pixels are selected. Is the target of the calculation, so that the pixels located in the range largely deviating from the original crop row position are excluded from the target of the calculation. As a result, it is possible to increase the detection accuracy of the approximate line segment obtained by the arithmetic processing, and at the same time reduce the number of pixels to be arithmetically operated, that is, the amount of information to be processed, and shorten the arithmetic time.

[0008]

As described above, the processing speed can be improved at the same time while ensuring the detection accuracy of the approximate line segment that is obtained corresponding to the crop row, and as a result, it can be used as steering control information when the work vehicle is automatically traveling. It can be used effectively.

[0009]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment in which the present invention is applied to a device for detecting the position of a seedling row planted in a field by a rice transplanter will be described below with reference to the drawings.

As shown in FIGS. 7 and 8, a seedling planting device 2 is provided to be movable up and down behind a machine body V in which both front wheels 1F and rear wheels 1R are steerable, and the seedlings are planted. With the device 2, the planted seedlings T as a plurality of crops arranged in rows are planted. Then, the color type image sensor S ₁ as an image pickup means for picking up an image of a field scene in a predetermined range including these already planted seedlings T is the body V.
Is provided on the front side of.

Explaining the mounting structure of the image sensor S ₁ , the base member is fixed to the seedling planting device 2 located at the rear of the machine V, and the support member 4 extending laterally to the front of the machine is provided. Image sensor S on the tip of
₁ is attached to the machine body V so as to capture an image of a row of planted seedlings adjacent to the machine body laterally outward from obliquely above. That is, in a state where the machine body V is properly along the row of the plurality of planted seedlings T arranged in the machine body traveling direction, the line segment L corresponding to the planted seedlings Tn adjacent to the unplanted side region is The image sensor S ₁ is arranged in such a state that it coincides with a traveling reference line La that passes through the center of the image pickup visual field in the front-rear direction. The seedling planting device 2 is installed on the mud surface of the field via the float 13, so that even if there is pitching or rolling of the machine body V, the effect is removed as much as possible. The image pickup posture with respect to is stabilized.

Then, a plurality of work strokes from one end side to the other end side of the field are set in a state of being arranged parallel to the machine lateral width direction, and in each work stroke, based on the image pickup information of the image sensor S _1. The steering control is carried out so that the vehicle automatically travels along the already planted seedling row. However, as it reaches the end of one work stroke, it is automatically turned in a state where the direction is changed by 180 degrees toward the start end of the next work stroke adjacent to the work stroke.

Therefore, the traveling direction of the machine body V with respect to the field is reversed every time the vehicle travels one stroke, and the position of the planted seedlings T with respect to the machine body V is laterally reversed.
One image sensor S ₁ is provided on each of the left and right sides of the machine body V, and the sensor on the side to be used is switched between the left and right for each stroke. In addition, in FIG. 7, the position of the planted seedling T is
Since it is on the right side of the airframe V, the image sensor S ₁ on the right side is used.

Explaining the structure of the machine body V, as shown in FIG. 1, the output of the engine E is transmitted to each of the front wheel 1F and the rear wheel 1R via a transmission device 5, and the transmission device 5 operates. as the operation state of the speed change operation state corresponds to a preset speed, the shifting state detection potentiometer R ₃ provided, and, on the basis of the detection information of the shifting state detecting potentiometer R _3, shift The electric motor 6 is driven.

Further, the front wheel 1F and the rear wheel 1R are constructed so that power steering is individually operated by hydraulic cylinders 7F and 7R, respectively, and steering angle detecting potentiometers R ₁ and R interlocked with the steering operation of the wheels. _The electromagnetically operated control valves 8F and 8R for operating the hydraulic cylinders 7F and 7R are driven so that the detected steering angle by ₂ becomes the target steering angle.

Therefore, a parallel steering system for steering the front wheels 1F and the rear wheels 1R at the same phase and at the same angle,
It is possible to selectively use three types of steering types: a four-wheel steering type in which the front wheels 1F and the rear wheels 1R are steered in opposite phases and at the same angle, and a two-wheel steering type in which only the front wheels 1F are turned. Is becoming

However, when the steering operation is automatically performed based on the image pickup information of the image sensor S ₁ , the two-wheel steering type is used, and when one work stroke is completed and the operation is moved to the next work stroke. The four-wheel steering type and the parallel steering type are used. In FIG. 1, S ₂ is a distance sensor for detecting the traveling distance based on the output speed of the transmission 5.

Next, a control configuration for approximating the line segment L corresponding to the boundary between the unplanted side seedling row and the already planted side seedling row based on the image pickup information of the image sensor S ₁ will be described. ..

As shown in FIG. 1, the image sensor S
₁ is configured to output the three primary color information R, G, B separately, and subtracts the blue information B that does not include the color component of the seedling T from the green information G that includes the color component of the seedling T, and is a binary value. The pixel Ta corresponding to the seedling T is extracted by the conversion.

In addition, a subtracter 9 for subtracting the blue color information B from the green color information G in the form of an analog signal,
A comparator 10 that binarizes the output of the subtractor 9 based on a preset threshold value corresponding to the color of the seedling T and outputs binarized information corresponding to the pixel Ta, and the output of the comparator 10. Corresponding to the image memory 11 that stores a signal as image information corresponding to a preset pixel density (32 × 32 pixels / 1 screen is set), and the seedling T stored in the image memory 11. Of the pixel information, only pixels Ta located within a predetermined number of pixels (± d pixels) from the pixel position on the screen of the line segment L ₀ previously obtained by the calculating unit 101 described later are selected (see FIG. 4), and these are selected. Of the selected pixel Ta as the object of calculation, information for approximating a line segment L connecting these pixels Ta to a straight line or a curve, and traveling control based on the information are controlled by a microcomputer. Each of the location 12 is provided.

That is, the subtractor 9 and the comparator 10 make the pixel T corresponding to the color of the seedling T as a crop.
This corresponds to the pixel extracting means 100 for extracting a, and using the control device 12 and the image memory 11, the line segment L ₀ of the pixel Ta of the pixel Ta previously obtained by the calculating means 101 described later is used. Pixel selecting means 102 for selecting only pixels Ta located within a predetermined number of pixels (± d pixels) from the pixel position on the screen, and the selected pixels T
Each of the calculation means 101 for obtaining the line segment L connecting a by approximating it to a straight line or a curve is configured.

Next, based on the flowchart shown in FIG. 2, the operation of the control device 12 will be described, and the configuration of each unit will be described in detail.

Every time the machine body V travels a set distance or every set time, the image pickup process by the image sensor S ₁ is executed and the pixel Ta corresponding to the seedling T is extracted by the pixel extracting means 100. Will be done.

To explain the extraction of the pixel Ta, since the color of the seedling T is green, when focusing on the green information G of the three primary color information of the field scene, the pixel of the seedling T is imaged. The value corresponding to is larger than the value of the pixel that has imaged another portion such as the mud surface of the field. However, since the natural light is almost totally reflected on the water surface, the reflected light contains all color components. Therefore, the value of the green color information G corresponding to the pixel of which the water surface is imaged is larger than the value of the pixel of which the other portion is imaged, like the pixel of which the seedling T is imaged. However, since the seedling T is greenish and the mud surface is brownish or grayish, the value of the blue component contained in the color component is low.

That is, paying attention to the blue color information B of the three primary color information of the image of the field, since the reflected light from the water surface contains all color components, the pixel value of the water surface becomes large, but The pixel value of T or the mud image is small.

Therefore, when the blue color information B is subtracted from the green color information G, only the value of the pixel corresponding to the seedling T becomes larger than the pixel value of the other part imaged, and the subtracted value is set as the set threshold value. By binarizing based on this, information of the pixel Ta corresponding to only the seedling T can be extracted (FIG. 4).
(See (a) and (b)). In the figure, eight pixels Ta ₁
~ Ta ₈ is shown in the extracted state. However, these 8
Each of the pixels Ta _{1 to} Ta ₈ may actually be one pixel in the pixel density (32 × 32 pixels / one screen), but may be one or more pixels.

Next, the line segment L ₀ previously obtained from the information of the pixels Ta ₁ to Ta ₈ corresponding to the plurality of seedlings T stored in the image memory 11 by the selection means 102. A predetermined number of pixels (± d
Only pixels Ta ₁ to Ta ₄ located within (pixels) are selected (see FIG. 4C). Here, since the vertical and vertical directions of the screen are controlled so that the vertical and vertical directions are along the seedling direction, that is, along the line segment L to be obtained, for the sake of simplification of the process, the screen of the previously obtained line segment L ₀ is displayed. The horizontal direction of the screen (x
Pixels located within the predetermined number of pixels (± d pixels) are selected along both sides of the (axis).

Next, the selected pixels Ta _{1 to} Ta _{4 are selected.}
The line segment L connecting the two is approximated to a straight line or a curved line by the calculation means 101. Here, the line segment L is calculated as a straight line by a Hough conversion process, which is a conversion unit for linear approximation.

The Hough transform will be described. As shown in FIG. 6, the x-axis passing through the center of the image pickup field of the image sensor S ₁ is used as a reference line in the polar coordinate system, and each pixel Ta is converted.
_A plurality of straight lines passing through _{1 to} Ta ₄ are set to the origin θ, that is, the center of the imaging field of view, with inclinations θ preset in a plurality of stages in the range of 0 to 180 degrees with respect to the x-axis based on the following formula (i). It is obtained as a combination with the corresponding distance ρ from the center of the screen. ρ = y · sin θ + x · cos θ …… (i)

After repeating the process of adding a two-dimensional histogram for counting the frequency of each straight line obtained until the value of the inclination θ set in the plurality of steps reaches 180 degrees for one pixel. , Each of the pixels Ta _{1 to} T
The frequencies of a plurality of types of straight lines passing through a ₄ will be counted for every pixel.

When the counting of the frequency of the straight line for each of the pixels Ta _{1 to} Ta ₄ is completed, the combination of the inclination θ and the distance ρ that has the maximum frequency is obtained from the value added to the two-dimensional histogram. One straight line Lx having the maximum frequency (see FIG. 6) is determined, and the straight line Lx is
The line segment L connecting the planted seedlings Tn adjacent to the unplanted side region on the image pickup surface of the image sensor S ₁ is obtained as linearly approximated information.

Next, the straight line Lx on the image pickup surface is stored in advance by the memory information of the shape and size of the image pickup field A of the image sensor S ₁ measured on the ground surface and the image pickup surface through which the straight line Lx having the maximum frequency passes. Based on the pixel positions a, b, and c at the positions (see FIG. 5 and FIG. 6). That is, as shown in FIG. 5, a straight line L on the ground surface that is set as a value of an inclination ψ with respect to a traveling reference line La that passes through the center of the imaging visual field A in the widthwise direction in the front-rear direction and a position δ in the widthwise direction. Will be converted into information.

To further explain, the front and rear positions (y = 16 and y =) of the imaging visual field A in the direction intersecting the straight line L corresponding to the line segment connecting the planted seedlings Tn adjacent to the unplanted side region.
-16) two side lengths l ₁ and l ₃₂ , the center of the screen (x = 1
6, the pixel position where y = 0), the length l ₁₆ in the lateral width direction of the imaging visual field A, and the distance h between the two front and rear sides.
Each of the above is measured in advance and stored in the control device 12.

The straight line Lx on the image pickup surface is
X-coordinate values X ₁ and X ₃₂ of positions a and b (positions where y = 16 and y = −16) of pixels intersecting the x-axis corresponding to two sides at the front and rear positions of the imaging visual field A, The value X ₁₆ of the x coordinate of the image position c at which the straight line Lx intersects the x axis passing through the center of the screen is obtained from the following equation (ii) which is a modification of the above equation (i). Xi = ([rho] -Yi * sin [theta]) / cos [theta] (ii) where Yi is 16,0, -16 respectively.

Then, based on the coordinate value on the x-axis obtained by the above equation (ii), the position δ in the lateral direction with respect to the traveling reference line La is calculated from the following equations (iii) and (iv).
And the slope ψ, the values of the calculated position δ and the slope ψ are
It is calculated as the position information of the straight line L corresponding to the above-mentioned seedling row on the ground surface.

[0036]

[Equation 1]

Therefore, in the steering control for automatically traveling the machine body V along the row of the planted seedlings T arranged in the machine body traveling direction, the inclination ψ of the straight line L with respect to the traveling reference line La and the lateral width direction are The steering operation is performed in a two-wheel steering manner so that the position δ and the position δ both approach zero.

The steering control will be described from the values of the inclination ψ of the straight line L with respect to the traveling reference line La and the position δ in the lateral width direction, and the current steering angle φ of the front wheels 1F. The target steering angle θf of the front wheel 1F is set based on the following equation (v), and the current steering angle φ detected by the front wheel steering angle detection potentiometer R ₁ is set to the target steering angle φf. The control valve 8F of the front wheel hydraulic cylinder 7F is driven so as to be maintained within the set dead zone with respect to the orientation angle θf. θf = K ₁ δ + K ₂ ψ + K ₃ Φ (v) Incidentally, K ₁ , K ₂ and K ₃ are constants preset according to the steering characteristics.

The turn control for reaching the end of one work stroke and turning toward the beginning of the next work stroke will be described. The travel distance detected by the distance sensor S ₂ is one work stroke. Although it will not be described in detail, when the set distance set corresponding to the length is exceeded, it is detected from the image information of the image pickup means S _{1 that} the end of the work stroke is reached. Then, when it is determined that the end of the work process has been reached, the planting work by the seedling planting device 2 is interrupted, the two-wheel steering system is switched to the four-wheel steering system, and the maximum cutting is performed during the set time. By maintaining the angle, the direction is changed 180 degrees to the next work stroke side, and then the parallel steering type is changed to maintain the maximum turning angle for the set time.
The position will be corrected in the lateral direction of the aircraft for the next work stroke, and the turn will be ended. After the end of the turn, the two-wheel steering system is restored, and the steering control in the next work stroke is restarted.

[Other Embodiments] In the above embodiment, the color image sensor S ₁ is used as the image pickup means for picking up the field scene, and the blue color information B is subtracted from the green color information G to obtain a binary value based on the set threshold value. The case where the pixel Ta corresponding to the seedling T is extracted by the conversion is illustrated. However, for example, by using all the three primary color information R, G, and B, their ratio corresponds to the color of the seedling T. A region having a set ratio range may be extracted as the pixel Ta, or a brightness signal of another portion such as a seedling portion and a mud surface may be obtained by using a monochrome image sensor S ₁ as the image pickup means. May be binarized based on an appropriate threshold value setting to extract pixels with a simple device, and the specific configuration of the pixel extracting means 100 can be variously changed.

Further, in the above embodiment, the support member 4 having the image sensor S ₁ attached to the front end is fixed to the seedling planting device 2 located at the rear of the machine body V at the base end, and the front end is fixed. Although it is configured to extend laterally to the front of the machine body, a structure may be used in which the support member 4 is fixed to a position on the front side of the machine body V.

Further, in the above embodiment, the pixel selection means 102.
In order to simplify the processing, the line segment L ₀ is located within a predetermined number of pixels (± d pixels) along both lateral directions (x-axis direction) of the screen from the pixel position of the line segment L ₀ obtained on the screen last time. Although it is configured to select pixels, strictly speaking, the traveling direction and the screen horizontal direction are not orthogonal to each other. Therefore, in order to perform pixel selection with higher accuracy, the pixel position on the screen of the line segment L ₀ is determined. It is also possible to select a pixel located within a predetermined number of pixels (± d pixels) in a direction orthogonal to the line segment L ₀ .

In the above embodiment, the Hough transform is used to linearly approximate the line segment L connecting the pixels Ta corresponding to the crop T. However, for example, the least square method is used. It is also possible to obtain information obtained by linear approximation or curve approximation, and the specific configuration of the arithmetic means 101 is as follows.
Various changes can be made (see Fig. 3).

Further, in the above embodiment, the present invention is applied to a device for automatically traveling a rice transplanter along a seedling row planted in a field, and an approximate line segment corresponding to the detected crop row. Although the information is exemplified as being configured to be used as control information for steering control, the present invention can be applied to an apparatus for detecting information on approximate line segments corresponding to various crop rows. Therefore, the specific configuration of each part such as the type of crop and the configuration of the traveling system of the machine body, and the usage pattern of the information of the approximate line segment corresponding to the detected crop sequence can be variously changed.

It should be noted that although reference numerals are given in the claims for convenience of comparison with the drawings, the present invention is not limited to the configurations of the accompanying drawings by the entry.

[Brief description of drawings]

FIG. 1 is a block diagram of a control configuration.

FIG. 2 is a flowchart of control operation.

FIG. 3 is a flowchart of control operation of another embodiment.

FIG. 4 is an explanatory diagram of image processing.

FIG. 5 is an explanatory diagram showing a relationship between a traveling direction of the body of the imaging field of view and an approximate straight line.

FIG. 6 is an explanatory diagram of Hough transform.

FIG. 7 is a schematic plan view of a rice transplanter.

FIG. 8 is a schematic side view of the same.

[Explanation of symbols]

L line segment L ₀ line segment S ₁ imaging means T crop Ta pixels Ta _{1 to} Ta ₄ pixels 100 pixel extraction means 101 calculation means 102 pixel selection means

Claims (1)

[Claims] 1. An image pickup means (S ₁ ) for picking up an image of a field in a predetermined range including a plurality of crops (T) arranged in a line, and the image pickup means (S ₁ ) for picking up image information based on the picked-up image information. A pixel extraction unit (100) for extracting a pixel (Ta) corresponding to a crop (T), and a line segment (L) connecting the pixel (Ta) based on the pixel information extracted by the pixel extraction unit (100). ) Is provided, and the line segment (previously obtained by the calculating means (101) among the pixel information extracted by the pixel extracting means (100). A pixel selection means (102) for selecting only pixels (Ta _{1 to} Ta ₄ ) located within a predetermined number of pixels from the pixel position of L ₀ ) on the screen is provided, and the calculation means (101) is provided.
Is a crop row detecting device configured to target the pixels (Ta _{1 to} Ta ₄ ) selected by the pixel selecting means (102) as an object of calculation.