JPH05165519A - Crop string detector - Google Patents

Abstract

PURPOSE:To secure the extraction accuracy at the time of extracting a straight line running along a crop string and also, to increase the processing speed by setting a selected straight line as an object of a processing. CONSTITUTION:A straight line selecting means 102 selects only a straight line which is in a relation being near a straight line extracted in the previous time by a straight line extracting means 103, in plural pieces of straight lines derived by a straight line arithmetic means 101. Subsequently, the straight line extracting means 103 sets the straight line selected by the straight line selecting means 102 as the object of the processing. In such a manner, extraction accuracy at the time of extracting a straight line running along the crop string can be secured and also, its processing speed can be increased.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image pickup means for picking up an image of a field scene in a predetermined range including a plurality of crops arranged in a row, and a pixel corresponding to the crop based on imaged image information by the image pickup means. And a straight line calculating means for obtaining a plurality of straight lines passing through the pixels extracted by the pixel extracting means and having inclinations set in a plurality of stages for each extracted pixel, The present invention relates to a crop row detecting device provided with a straight line extracting unit that extracts a straight line having the maximum frequency among a plurality of straight lines obtained by the straight line calculating unit.

[0002]

2. Description of the Related Art A crop row detecting device of the above type has a structure in which, when a seedling as a crop is planted in a field in set intervals at a set unit, such as a rice planting machine, the machines are aligned in the machine traveling direction. In order to automatically drive along the seedling row, the position and direction of the existing seedling row with respect to the machine position are detected to obtain control information. Therefore, for example, by using Hough transform, etc., after calculating and obtaining a plurality of straight lines that pass through each pixel extracted corresponding to the seedling on the imaging screen and have inclinations set in multiple stages The line having the maximum frequency obtained by cumulatively adding the frequencies of the plurality of lines was extracted as the line along the row of planted seedlings.

Conventionally, for example, when a plurality of straight lines passing through the pixels extracted corresponding to the seedlings on the image pickup screen are calculated and the frequencies of the straight lines are cumulatively added, for example, calculation is performed. The above addition process was performed for all the straight lines without any restrictions on the position and inclination of the obtained straight lines on the screen.

[0004]

However, in the above-mentioned conventional technique, all the straight lines passing through the pixels extracted corresponding to the seedlings on the image pickup screen are subject to the addition process.
A straight line whose position or inclination on the screen greatly deviates from the straight line along the originally planted seedling row, which is the target of processing,
Due to the mixing of these unnecessary information, there is a possibility that a straight line other than the one that should be originally obtained becomes the maximum frequency and may be erroneously extracted, and the amount of information that is the target of the addition process increases and the calculation time becomes long, and There is also a disadvantage that the control delay becomes large when the work vehicle is used as steering control information for automatically traveling along the crop row.

The present invention has been made in view of the above circumstances, and an object thereof is to ensure the extraction accuracy when extracting a straight line along a crop row and also to improve the processing speed thereof. It is to provide a crop row detecting device capable of performing the above.

[0006]

A first characteristic configuration of a crop row detecting device according to the present invention is a straight line previously extracted by the straight line extracting means from among a plurality of straight lines obtained by the straight line calculating means. A straight line selecting means for selecting only straight lines having a close relationship is provided, and the straight line extracting means is configured to process the straight lines selected by the straight line selecting means.

In a second characteristic configuration, the straight line calculating means is configured to convert the plurality of straight lines into parameters representing the position and inclination of each straight line on the screen by Hough transformation, and The selecting means selects only a straight line having a parameter value within a setting range centered on the parameter value of the previously extracted straight line from among the plurality of straight lines converted to the parameter by the straight line calculating means. It is in the point that is configured.

[0008]

According to the first characteristic configuration of the present invention, a straight line corresponding to the original crop sequence extracted in the previous extraction process and a straight line having a close value on the screen, such as a position or a tilt, are present. Only the selected straight lines are excluded, and the straight lines corresponding to the above-mentioned original straight lines corresponding to the position and inclination on the screen are excluded, and the addition process is performed only on these selected straight lines. Will be seen. As a result, straight lines other than the straight lines to be originally obtained will not be erroneously extracted with the maximum frequency, and at the same time, the number of straight lines to be subjected to addition processing, that is, the amount of information to be processed will be small and the calculation time Shortened.

According to the second characteristic configuration, a plurality of straight lines passing through the pixels extracted corresponding to the position of the crop are converted into parameters representing the position and inclination of each straight line on the screen by Hough transform. After that, only the straight lines having the parameter values within the setting range centered on the parameter values of the straight lines corresponding to the original crop row extracted in the previous extraction process are added, and these selected straight lines are added. Processing will be performed. As a result, straight line parameters other than the straight line parameters to be originally obtained are prevented from being erroneously extracted with the maximum frequency, and at the same time, the number of straight lines subject to addition processing, that is, the amount of information to be processed is reduced. Calculation time is shortened.

[0010]

As described above, according to the first characteristic configuration, it is possible to secure the extraction accuracy when extracting the straight line along the crop row, and also to improve the processing speed by shortening the calculation time. Therefore, it can be effectively used as steering control information when the work vehicle is automatically traveling.

According to the second characteristic configuration, the effectiveness of using the Hough transform as a method for obtaining the straight line along the crop row can be further enhanced.

[0012]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment in which the present invention is applied to a device for detecting the position of a seedling row planted in a field by a rice transplanter will be described below with reference to the drawings.

As shown in FIGS. 7 and 8, a seedling planting device 2 is provided up and down to the rear of a machine body V in which both the front wheels 1F and the rear wheels 1R are steerable, and the seedling planting is performed. With the device 2, the planted seedlings T as a plurality of crops arranged in a row are planted. Further, a color type image sensor S ₁ as an image pickup means for picking up an image of a farm field in a predetermined range including these planted seedlings T is provided on the front side of the machine body V.

Explaining the mounting structure of the image sensor S ₁ , the base member is fixed to the seedling planting device 2 located at the rear of the machine body V, and the support member 4 is extended laterally to the front side of the machine body. Image sensor S on the tip of
₁ is attached to the machine body V so as to capture an image of a row of planted seedlings adjacent to the machine body laterally outward from obliquely above. That is, in a state in which the machine body V is properly aligned with the row of the plurality of planted seedlings T arranged along the machine body traveling direction, the straight line L corresponding to the planted seedlings Tn adjacent to the unplanted side region is The state is such that it coincides with the traveling reference line La that passes through the center of the image pickup field of the image sensor S _{1 in} the front-rear direction. The seedling planting device 2 is installed on the mud surface of the field via the float 13, so that even if there is pitching or rolling of the machine body V, the effect is removed as much as possible. The image pickup posture with respect to is stabilized.

A plurality of work strokes from one end side to the other end side of the field are set in a state of being arranged parallel to the lateral direction of the machine body. In each work stroke, based on the image pickup information of the image sensor S _1. The steering control is carried out so that the vehicle automatically travels along the already planted seedling row. However, as it reaches the end of one work stroke, it is automatically turned in a state where the direction is changed by 180 degrees toward the start end of the next work stroke adjacent to the work stroke.

Therefore, the traveling direction of the machine body V with respect to the field is reversed every time the vehicle travels one stroke, and the position of the planted seedlings T with respect to the machine body V is left-right reversed.
One image sensor S ₁ is provided on each of the left and right sides of the machine body V, and the sensor on the side to be used is switched between the left and right for each stroke. In addition, in FIG. 7, the position of the planted seedling T is
Since it is on the right side of the airframe V, the image sensor S ₁ on the right side is used.

Explaining the structure of the machine body V, as shown in FIG. 1, the output of the engine E is transmitted to each of the front wheel 1F and the rear wheel 1R via a transmission device 5, and the transmission device 5 operates. as the operation state of the speed change operation state corresponds to a preset speed, the shifting state detection potentiometer R ₃ provided, and, on the basis of the detection information of the shifting state detecting potentiometer R _3, shift The electric motor 6 is driven.

Further, the front wheel 1F and the rear wheel 1R are constructed so that power steering is individually operated by hydraulic cylinders 7F and 7R, respectively, and steering angle detecting potentiometers R ₁ and R interlocked with the steering operation of the wheels. _The electromagnetically operated control valves 8F and 8R for operating the hydraulic cylinders 7F and 7R are driven so that the detected steering angle by ₂ becomes the target steering angle.

Therefore, a parallel steering system for steering the front wheels 1F and the rear wheels 1R in the same phase and at the same angle,
It is possible to selectively use three types of steering types: a four-wheel steering type in which the front wheels 1F and the rear wheels 1R are steered in opposite phases and at the same angle, and a two-wheel steering type in which only the front wheels 1F are turned. Is becoming

However, when the steering operation is automatically performed based on the image pickup information of the image sensor S ₁ , the two-wheel steering type is used, and when one work process is finished and the next work process is performed, The four-wheel steering type and the parallel steering type are used. In FIG. 1, S ₂ is a distance sensor for detecting the traveling distance based on the output speed of the transmission 5.

Next, a control configuration for approximating the straight line L corresponding to the boundary between the unplanted side seedling row and the already planted side seedling row based on the image pickup information of the image sensor S ₁ will be described.

As shown in FIG. 1, the image sensor S
₁ is configured to output the three primary color information R, G, B separately, and subtracts the blue information B that does not include the color component of the seedling T from the green information G that includes the color component of the seedling T, and is a binary value. The pixel Ta corresponding to the seedling T is extracted by the conversion.

In addition, a subtracter 9 for subtracting the blue color information B from the green color information G in the state of an analog signal,
A comparator 10 that binarizes the output of the subtractor 9 based on a preset threshold value corresponding to the color of the seedling T and outputs binarized information corresponding to the pixel Ta, and the output of the comparator 10. Corresponding to the image memory 11 that stores a signal as image information corresponding to a preset pixel density (32 × 32 pixels / 1 screen is set), and the seedling T stored in the image memory 11. Based on the pixel information, a plurality of straight lines passing through the pixel Ta and having inclinations set in a plurality of steps are obtained for each pixel Ta, and a straight line extracting means 10 to be described later among the plurality of straight lines is obtained.
In step 3, only the straight line L ₀ obtained last time and the line whose position and inclination are close to each other on the screen are selected, and the straight line L having the maximum frequency is extracted from these selected straight lines, and the extracted straight line L Each of the control devices 12 using a microcomputer for controlling the traveling based on the information is provided.

That is, the subtractor 9 and the comparator 10 make the pixel T corresponding to the color of the seedling T as a crop.
This corresponds to the pixel extracting means 100 for extracting a, and the control device 12 and the image memory 11 are used to pass a plurality of pixels which pass through the pixel Ta and have inclinations set in a plurality of stages. A straight line calculating means 101 for obtaining a straight line by converting it into parameters ρ and θ representing the position and inclination on the screen by Hough transform, and a straight line extraction described later among the plurality of straight lines converted into the parameters ρ and θ. Means 1
Parameter ρ ₀ of the straight line L ₀ previously extracted by 03,
Setting range centered on the value of θ ₀ (ρ ₀ ± ρ _d , θ ₀ ± θ
_The straight line selecting means 102 for selecting only the straight lines having the values of the parameters ρ and θ in _d ) (see FIG. 3), and the parameter ρ having the maximum frequency among the set of the parameters ρ and θ of the selected straight lines. , Θ, the straight line extracting means 103 for extracting the straight line L having the respective values.

Next, the operation of the controller 12 will be described based on the flowchart shown in FIG.

Every time the machine body V travels a set distance or every set time, the image pickup process by the image sensor S ₁ is executed and the pixel Ta corresponding to the seedling T is extracted by the pixel extracting means 100. Will be done.

To explain the extraction of the pixel Ta, since the color of the seedling T is green, when focusing on the green information G of the three primary color information of the image of the field, the pixel of the seedling T is imaged. The value corresponding to is larger than the value of the pixel that has imaged another portion such as the mud surface of the field. However, since the natural light is almost totally reflected on the water surface, the reflected light contains all color components. Therefore, the value of the green color information G corresponding to the pixel of which the water surface is imaged is larger than the value of the pixel of which the other portion is imaged, like the pixel of which the seedling T is imaged. However, since the seedling T is greenish and the mud surface is brownish or grayish, the value of the blue component contained in the color component is low.

That is, paying attention to the blue color information B of the three primary color information of the image of the field, since the reflected light from the water surface contains all the color components, the value of the pixel of which the water surface is imaged is large. The pixel value of T or the mud image is small.

Therefore, when the blue color information B is subtracted from the green color information G, only the value of the pixel corresponding to the seedling T becomes larger than the pixel value of the other part imaged, and the subtracted value is set as the set threshold value. By binarizing based on this, information of the pixel Ta corresponding to only the seedling T can be extracted (FIG. 4).
(See (a) and (b)). In the figure, eight pixels Ta ₁
~ Ta ₈ is shown in the extracted state. However, these 8
Each of the pixels Ta _{1 to} Ta ₈ may actually be one pixel in the pixel density (32 × 32 pixels / one screen), but may be one or more pixels.

Next, each of the extracted pixels Ta _{1 to} Ta is extracted.
_A plurality of straight lines passing through ₈ are obtained by converting them into parameters ρ and θ representing the position and inclination on the screen by Hough transform by the straight line calculating means 101.

The Hough transform will be described. As shown in FIG. 6, the x-axis passing through the center of the field of view of the image sensor S ₁ is used as the reference line in the polar coordinate system, and each pixel Ta is converted.
_A plurality of straight lines passing through _{1 to} Ta ₈ are set to the origin θ, that is, the center of the imaging visual field, with inclinations θ preset in a plurality of stages in the range of 0 to 180 degrees with respect to the x-axis based on the following formula (i). It is obtained as a combination with the corresponding distance ρ from the center of the screen. ρ = y · sin θ + x · cos θ …… (i)

However, among the combinations of the parameters ρ and θ obtained above, a setting range (ρ ₀ ± ρ _d , θ) centered on the values of the parameters ρ ₀ and θ ₀ of the straight line L ₀ extracted last time
Only the straight lines having the values of the parameters ρ and θ within ₀ ± θ _d ) are sorted by the straight line sorting means 102.

Next, with respect to all the pixels constituting each of the pixels Ta _{1 to} Ta ₈ , the process of adding the two-dimensional histogram to the frequencies of the parameters ρ and θ of each straight line obtained by the Hough transform as described above is performed. Repeat for each pixel Ta
The frequencies of a plurality of types of straight lines passing through _{1 to} Ta ₈ are counted for all pixels.

When the counting of the frequency of the straight line for each of the pixels Ta _{1 to} Ta ₈ is completed, the combination of the inclination θ and the distance ρ that has the maximum frequency is obtained from the value added to the two-dimensional histogram. One straight line Lx having the maximum frequency (see FIG. 6) is determined, and the straight line Lx is
A straight line L connecting the planted seedlings Tn adjacent to the unplanted side region on the image pickup surface of the image sensor S ₁ is obtained as linearly approximated information.

Next, the straight line Lx on the image pickup surface is stored in advance by the memory information of the shape and size of the image pickup field A of the image sensor S ₁ measured on the ground surface, and the image pickup surface through which the straight line Lx of the maximum frequency passes. Based on the pixel positions a, b, and c at the positions (see FIG. 5 and FIG. 6). That is, as shown in FIG. 5, a straight line L on the ground surface that is set as a value of an inclination ψ with respect to a traveling reference line La that passes through the center of the imaging visual field A in the widthwise direction in the front-rear direction and a position δ in the widthwise direction. Will be converted into information.

To further explain, the front and rear positions (y = 16 and y =) of the imaging visual field A in the direction intersecting the straight line L corresponding to the line segment connecting the planted seedlings Tn adjacent to the unplanted side region.
-16) two side lengths l ₁ and l ₃₂ , the center of the screen (x = 1
6, the pixel position where y = 0), the length l ₁₆ in the lateral width direction of the imaging visual field A, and the distance h between the two front and rear sides.
Each of the above is measured in advance and stored in the control device 12.

Then, the straight line Lx on the imaging plane is
X-coordinate values X ₁ and X ₃₂ of positions a and b (positions where y = 16 and y = −16) of pixels intersecting the x-axis corresponding to two sides at the front and rear positions of the imaging visual field A, The value X ₁₆ of the x coordinate of the image position c at which the straight line Lx intersects the x axis passing through the center of the screen is obtained from the following equation (ii) which is a modification of the above equation (i). Xi = ([rho] -Yi * sin [theta]) / cos [theta] (ii) where Yi is 16,0, -16 respectively.

Then, based on the coordinate values on the x-axis obtained by the above equation (ii), from the following equations (iii) and (iv), the position δ in the lateral direction with respect to the traveling reference line La is calculated.
And the slope ψ, the values of the calculated position δ and the slope ψ are
It is calculated as the position information of the straight line L corresponding to the above-mentioned seedling row on the ground surface.

[0039]

[Equation 1]

Therefore, in the steering control for automatically traveling the machine body V along the row of the planted seedlings T arranged in the machine body traveling direction, the inclination ψ of the straight line L with respect to the traveling reference line La and the lateral direction The steering operation is performed in a two-wheel steering manner so that the position δ and the position δ both approach zero.

The steering control will be described from the values of the inclination ψ of the straight line L with respect to the traveling reference line La and the position δ in the lateral width direction, and the current steering angle φ of the front wheels 1F. The target steering angle θf of the front wheel 1F is set based on the following equation (v), and the current steering angle φ detected by the front wheel steering angle detection potentiometer R ₁ is set to the target steering angle φf. The control valve 8F of the front wheel hydraulic cylinder 7F is driven so as to be maintained within the set dead zone with respect to the orientation angle θf. θf = K ₁ δ + K ₂ ψ + K ₃ Φ (v) Incidentally, K ₁ , K ₂ and K ₃ are constants preset according to the steering characteristics.

The turn control for reaching the end of one work stroke and turning toward the start of the next work stroke will be described. The travel distance detected by the distance sensor S ₂ is one work stroke. Although it will not be described in detail, when the set distance set corresponding to the length is exceeded, it is detected from the image information of the image pickup means S _{1 that} the end of the work stroke is reached. Then, when it is determined that the end of the work process has been reached, the planting work by the seedling planting device 2 is interrupted, the two-wheel steering system is switched to the four-wheel steering system, and the maximum cutting is performed during the set time. By maintaining the angle, the direction is changed 180 degrees to the next work stroke side, and then the parallel steering type is changed to maintain the maximum turning angle for the set time.
The position will be corrected in the lateral direction of the aircraft for the next work stroke, and the turn will be ended. After the end of the turn, the two-wheel steering system is restored, and the steering control in the next work stroke is restarted.

[Other Embodiment] In the above embodiment, the color type image sensor S ₁ is used as the image pickup means for picking up a field scene, the blue color information B is subtracted from the green color information G, and the binary information is binarized based on the set threshold value. The case where the pixel Ta corresponding to the seedling T is extracted by the conversion is illustrated. However, for example, by using all the three primary color information R, G, and B, their ratio corresponds to the color of the seedling T. A region having a set ratio range may be extracted as the pixel Ta, or a brightness signal of another portion such as a seedling portion and a mud surface may be obtained by using a monochrome image sensor S ₁ as the image pickup means. May be binarized based on an appropriate threshold value setting to extract pixels with a simple device, and the specific configuration of the pixel extracting means 100 can be variously changed.

Further, in the above embodiment, the support member 4 having the image sensor S ₁ attached to the front end is fixed to the seedling planting device 2 located at the rear of the machine body V, and the front end is fixed. Although it is configured to extend laterally to the front of the machine body, a structure may be used in which the support member 4 is fixed to a position on the front side of the machine body V.

Further, in the above embodiment, the straight line calculation means 101
By means of Hough transformation, and means 102 for selecting only a straight line having a close relationship with the straight line L ₀ previously extracted by the pixel extracting means 103, with respect to the straight line parameters ρ and θ obtained by the Hough transformation. Although the range is set, as shown in FIG. 9, the straight line calculating means 101 is represented by a straight line equation y = p * x + r on the xy coordinate axes on the screen, and the slope p of the straight line or You may make it set the range with respect to the intercept r which is a position which intersects with ay coordinate axis. In this case, the count of the two-dimensional histogram is added for the combination of the slope p of the straight line and the intercept r.

In the above embodiment, the present invention is applied to a device for automatically traveling a rice transplanter along a seedling row planted in a field, and an approximate line segment corresponding to the detected crop row. Although the information is exemplified as being configured to be used as control information for steering control, the present invention can be applied to an apparatus for detecting information on approximate line segments corresponding to various crop rows. Therefore, the specific configuration of each part such as the type of crop and the configuration of the traveling system of the machine body, and the usage pattern of the information of the approximate line segment corresponding to the detected crop sequence can be variously changed.

It should be noted that reference numerals are added to the claims for convenience of comparison with the drawings, but the present invention is not limited to the structures of the accompanying drawings by the entry.

[Brief description of drawings]

FIG. 1 is a block diagram of a control configuration.

FIG. 2 is a flowchart of control operation.

FIG. 3 is an explanatory diagram of straight line selection.

FIG. 4 is an explanatory diagram of pixel extraction.

FIG. 5 is an explanatory diagram showing a relationship between a traveling direction of the body of the imaging field of view and an approximate straight line.

FIG. 6 is an explanatory diagram of Hough transform.

FIG. 7 is a schematic plan view of a rice transplanter.

FIG. 8 is a schematic side view of the same.

FIG. 9 is a flowchart of control operation of another embodiment.

[Explanation of symbols]

L straight line L ₀ straight line S ₁ imaging means T crop Ta pixel ρ, θ parameter ρ ₀ , θ ₀ parameter ρ ₀ ± ρ _d , θ ₀ ± θ _d setting range 100 pixel extraction means 101 straight line calculation means 102 straight line selection means 103 straight line Extraction means

Claims (2)

[Claims] 1. An image pickup means (S ₁ ) for picking up an image of a field in a predetermined range including a plurality of crops (T) arranged in a line, and the image pickup means (S ₁ ) for picking up image information based on the picked-up image information. A pixel extraction unit (100) that extracts a pixel (Ta) corresponding to a crop (T), and a tilt that passes through the pixel (Ta) extracted by the pixel extraction unit (100) and is set in a plurality of stages. Pixels (T
The straight line calculating means (101) for each a) and the straight line extracting means (103) for extracting the straight line (L) having the maximum frequency among the plurality of straight lines obtained by the straight line calculating means (101) are provided. The crop row detecting device according to the present invention, which has a close relationship with a straight line (L ₀ ) previously extracted by the straight line extracting means (103) among a plurality of straight lines obtained by the straight line calculating means (101). A straight line selecting means (102) for selecting only straight lines is provided, and the straight line extracting means (103) is configured to target the straight lines selected by the straight line selecting means (102) for processing. Detection device. 2. The crop row detecting device according to claim 1.
Then, the straight line calculation means (101) detects the straight lines.
Parameter that represents the position and slope of each straight line on the screen
Configured to convert to meters (ρ, θ)
In addition, the straight line selecting means (102) is the straight line calculating means.
Converted to the above parameters (ρ, θ) by (101)
The previously extracted line (L₀)
Parameter of (ρ₀, Θ₀) Centered on the setting range
(Ρ₀± ρ _d, Θ₀± θ_dParameters in () (ρ, θ)
Configured to select only straight lines with values
Crop row detector.