JP3044141B2 - Planting condition detector for crop planting machines - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a planting state detecting device for a crop planting machine.

[0002]

2. Description of the Related Art Hitherto, the planting state of a crop has been visually judged by an operator. By the way, for example, when a rice planting machine as a crop planting machine is automatically run without human intervention, when planting seedlings as crops in a field in a row, the planting state of the seedlings to be planted with the progress of the machine body is also reduced. It is necessary to detect whether or not the plant is normal, that is, the occurrence of a missing plant (it becomes a seedling jam if it is continued in a certain row) or a shift of the planting position. The operator judged the condition with the naked eye.

[0003]

Therefore, in the prior art, it is troublesome for the operator to judge the planting condition of the crop, so that an abnormal occurrence of the planting condition such as lack of plant may be overlooked. In this case, the planting operation is continued in an abnormal state, and there is a possibility that planting failure such as lack of a plant or displacement of the planting position occurs frequently in the whole field. Also,
For example, when the operator manually operates the rice transplanter, it is necessary to determine the planting state at the same time as the operation, and in this case, a greater burden is imposed on the operator, and the risk of occurrence of the above-mentioned oversight and the like is large. there were.

[0004] The present invention has been made in view of the above-mentioned circumstances, and an object thereof is to provide a method for planting a crop such as a seedling in a field while running a crop planting machine such as a rice planting machine.
An object of the present invention is to provide a planting state detecting device capable of accurately and quickly detecting the planting state of a crop to be planted with the progress of an airframe without human intervention.

[0005]

A first characteristic configuration of a planting state detecting device for a crop planting machine according to the present invention is that, in a field scene on the rear side in the traveling direction of the aircraft, the crop is moved along with the traveling of the aircraft. An imaging unit that images a predetermined range including a portion where a crop to be planted is to be planted; an area extraction unit that extracts an area corresponding to the crop based on imaging information of the imaging unit; And a determining means for determining the planting state of the crop.

According to a second characteristic configuration, in addition to the first characteristic configuration, an information storage means for storing information of the discriminating means in a time-series manner is provided, and the discriminating means is provided in the storage of the information storage means. The point is that the planting state is determined based on information.

[0007]

According to the first feature configuration of the present invention, a field scene in a predetermined range on the rear side in the advancing direction of the fuselage including the crop planting location where the crop is to be planted with the advancing of the fuselage is imaged. From the area information extracted corresponding to the crop from the information, the planting state of the crop, for example, whether or not there is a planting defect such as a lack of plant or a shift in the planting position is determined.

[0008] According to the second characteristic configuration, discrimination information on the planting status of the crop is stored in a time-series manner,
From this time-series stored information, it is possible to detect, for example, a row or the like in which seedlings are continuously generated due to continuous lack of stock.

[0009]

According to the first aspect of the present invention, since the planting state immediately after planting is detected, the occurrence of planting failure can be detected accurately and promptly. By stopping and eliminating the cause of the poor planting, the above-mentioned problem in the case where the work is continued in the poor planting state is avoided, and at the same time, it is not necessary for the operator to visually determine the planting state as in the conventional case. Thus, the burden on the worker is reduced.

Further, according to the second characteristic configuration, it is possible to detect a case in which planting failures such as clogging of seedlings continue in time with a smaller storage capacity than in the case of storing the extracted information, for example. Thus, the effect of the first characteristic configuration can be further enhanced while maintaining a simple configuration.

[0011]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment in which the present invention is applied to a device for detecting a planting state of a seedling as a crop planted in a field by a rice planter as a crop planting machine will be described below with reference to the drawings.

As shown in FIGS. 2 to 4, a seedling planting device 2 is provided at the rear side of an airframe V having both front wheels 1F and rear wheels 1R operable for steering operation. In the planting device 2, a row (6 in this example)
A plurality of seedlings T as crops arranged in a row are planted. The color image sensor S1 for imaging a field scene on the front side in the machine traveling direction in a predetermined range including the planted seedlings Tn located adjacent to the side of the machine V among the already planted seedlings T is provided by the machine body V. Is provided on the front side.
Also, in the field scene on the rear side in the traveling direction of the aircraft, scheduled seedling planting locations T1 to which seedlings are to be planted as the aircraft V progresses.
A color image sensor S3 as an imaging unit for imaging a predetermined range including T6 is provided on the rear side of the body V.

The mounting structure of the image sensor S1 on the front side of the fuselage will be described. The base end is fixed to the front side of the fuselage V, and the image sensor is mounted on the distal end of the support member 4 extending laterally of the fuselage. S <b> 1 is attached, and is provided so as to image a row of planted seedlings adjacent to the body V on the lateral outside of the body from diagonally above. That is, in the state where the body V is properly aligned with the row of the plurality of planted seedlings T arranged in the body traveling direction, the unplanted side area N
The line segment L corresponding to the already planted seedling Tn adjacent to is set so as to coincide with the running reference line La passing through the center of the field of view of the image sensor S1 in the front-rear direction. In addition, the mounting structure of the image sensor S3 on the rear side of the fuselage will be described. The support member 13 has a base end fixed at a substantially central position in the lateral direction of the fuselage V at the rear side of the fuselage V and extends rearward of the fuselage. An image sensor S3 is attached to the distal end portion, and is provided so as to capture an image of an already-planted seedling row arranged in a row on the rear side of the machine body V from obliquely above.

A plurality of work steps from one end to the other end of the field are set in a state of being arranged in parallel in the machine width direction. In each work step, based on image information of the image sensor S1, The steering control is performed so as to automatically travel along the already planted seedling row. However, as the end of one work stroke is reached, the vehicle is automatically turned in a state of turning 180 degrees toward the start end of the next work stroke adjacent to the work stroke. Therefore, the traveling direction of the body V with respect to the field is reversed every time the vehicle V travels one stroke, and the position of the planted seedlings T with respect to the body V is in a state of being horizontally reversed.
Is provided for each of the right and left sides of the machine V, and the sensor on the side to be used is switched left and right for each stroke. still,
In FIG. 2, since the position of the planted seedling T is on the right side of the machine body V, the right image sensor S1 is used.

In the seedling planting apparatus 2, as shown in FIG. 4, the seedlings T are supplied from above in a state where they are aligned and held on the seedling sheet 16, and the supplied seedlings T at the lowermost position are supplied.
Each plant is scraped from above by a rotating planting claw 15 and planted in a field. or,
The rotation shaft portion of the planting claw 15 is provided with a planting phase detection sensor S4 using a switch for detecting the rotation phase of the planting claw 15. The information of the planting phase detection sensor S4 is used as imaging timing information of the image sensor S3 on the rear side of the machine body, as described later.
It is input to the control device 12 described later.

The structure of the machine body V will be described. As shown in FIG. 1, the output of the engine E is transmitted to each of the front wheel 1F and the rear wheel 1R via a transmission 5, and the transmission 5 as the operation state of the speed change operation state corresponds to a preset speed, it is provided shifting state detecting potentiometer R _3. And
A control device 12 using a microcomputer is provided,
The control device 12, based on the shifting state detection information of the detecting potentiometer R _3, and is configured to drive the electric motor for speed change use 6.

[0017] Further, the front wheel 1F and the rear wheel 1R are respectively hydraulic cylinders 7F, is configured to be powered steering operation to each other by 7R, potentiometer R ₁ for a steering angle detection interlocked with the steering operation of the wheel, R _The control device 12 drives the electromagnetically operated control valves 8F, 8R for operating the hydraulic cylinders 7F, 7R based on the information of _{2 so that} the detected steering angle becomes the target steering angle. Have been. Therefore, a parallel steering system in which the front wheel 1F and the rear wheel 1R are steered in phase and at the same angle, a four-wheel steering system in which the front wheel 1F and the rear wheel 1R are steered in opposite phase and at the same angle, and only the front wheel 1F It is possible to select and use three types of steering types, that is, a two-wheel steering type in which the direction is changed. However, when the steering operation is automatically performed based on the imaging information of the image sensor S1, the aforementioned 2
In addition to using the wheel steering system, when one work cycle is completed and the process moves to the next work cycle, the four-wheel steering system or the parallel steering system is used. In FIG. 1, S2 is a distance sensor for detecting a traveling distance based on the output rotation speed of the transmission 5.

Next, a control configuration for approximating the line segment L corresponding to the boundary between the unplanted area N and the existing area M based on the imaging information of the image sensor S1 will be described. As shown in FIG.
Numeral 1 is configured to output three primary color information R, G, and B separately, and subtracts blue information B not including the color component of the seedling T from green information G including the color component of the seedling T. The binarization is configured to extract a region Ta corresponding to the seedling T (see FIG. 5). In addition, since the color of the seedling T is green, focusing on green information G among the three primary color information obtained by imaging the field scene, the value corresponding to the pixel at which the seedling T is imaged is the mud of the field. The value is larger than the value of a pixel that has imaged another part such as a surface. However, since natural light is almost totally reflected on the water surface, the reflected light contains all color components. Therefore, the value of the green information G corresponding to the pixel capturing the water surface is larger than the value of the pixel capturing the other portion, similarly to the pixel capturing the seedling T.
When attention is paid to blue information B among the three primary color information obtained by imaging the field scene, since the seedling T is greenish and the mud surface is brownish or grayish, it is included in the color component. The value of the blue component is low. However, since the reflected light from the water surface contains all color components, the value of the pixel that images the water surface is large. Then, when the blue information B is subtracted from the green information G, only the value of the pixel corresponding to the seedling T becomes larger than the value of the pixel obtained by imaging the other portion. When the binarization is performed based on
That is, it is possible to extract a region corresponding only to this.

A subtractor 9 for subtracting the blue information B from the green information G in the form of an analog signal, and outputs the output of the subtracter 9 based on a preset threshold value corresponding to the color of the seedling T. Comparator 10 for binarizing and outputting binarized information corresponding to region Ta, and comparator 1
Each of the image memories 11 is provided for storing an output signal of 0 as pixel information corresponding to a preset pixel density (32 × 32 pixels / 1 screen), and corresponds to a seedling T stored in the image memory 11. The control device 12 is configured to determine the line segment L as information approximated to a straight line or a curve based on the pixel information, and to control the traveling based on the information. In other words, the subtractor 9 and the comparator 10 correspond to the area extracting means 100 for extracting the area Ta corresponding to the color of the seedling T, and the area extracting means using the control device 12. Calculating means for calculating the line segment L connecting the area Tan corresponding to the seedling Tn adjacent to the non-planting side area N in these areas Ta by using the area Ta extracted by 100 as an object of calculation. 102 is configured.

Next, the image sensor S on the rear side of the fuselage
A control configuration for determining the planting state of the seedlings T on the rear side of the machine body that is planted with the progress of the machine body V based on the imaging information of No. 3 will be described. As shown in FIG. 1, the image sensor S3 is configured to separately output the three primary color information R, G, and B, and, similarly to the case of the boundary detection processing, from the green information G to the blue information B. Subtract 2
It is configured to extract a region Ta corresponding to the seedling T by converting the value (see FIG. 9). That is,
Similarly to the area extracting means 100, the subtractor 9 and the comparator 10 constitute an area extracting means 200 for extracting an area Ta corresponding to the seedling T based on the image information of the image sensor S3. 10 output signals at a preset pixel density (32 × 32 pixels /
An image memory 21 for storing pixel information corresponding to one screen) is provided, and a judging means for judging the planting state of the seedlings T based on the information of the area extracting means 200 using the control device 12. 101 is configured.
Further, the image memory 21 also functions as an information storage unit that stores the discrimination information of the discrimination unit 101 in chronological order, and the discrimination unit 101 stores the seedling T based on the storage information of the image memory 21. It is configured to determine the planting state.

Next, based on the flowcharts shown in FIGS. 8 and 12, the operation of the control device 12 will be described, and the boundary detection processing for the boundary L and the planting state detection processing for determining the planting state of the seedlings T will be described. The configuration of each unit that executes the above will be described in detail.

In the boundary detection process (FIG. 8), every time the vehicle travels for a set distance or for a set time, an image pickup process is executed by the image sensor S1. The corresponding area Ta is extracted. Then, of the extracted areas Ta, areas Tan adjacent to the non-planting side area N are extracted. In each of these areas Tan, the nearest pixel to the stock source side is extracted as a representative point P, and these representative points P Is approximated to a straight line or a curve by the calculation means 102. Here, the line segment L is calculated as a straight line by a Hough transform process, which is a straight line approximation converting means.

The Hough transform will now be described. As shown in FIG. 6, the x-axis passing through the center of the field of view of the image sensor S1 is used as a reference line in a polar coordinate system to form each of the regions T.
A plurality of straight lines passing through a are defined on the basis of the following equation (i), a gradient θ previously set in a plurality of steps in a range of 0 to 180 degrees with respect to the x-axis, and a screen corresponding to the origin, that is, the center of the imaging visual field. It is obtained as a combination with the distance ρ from the center. ρ = y · sin θ + x · cos θ (i)

Then, for one pixel in each region Ta, a process of adding a two-dimensional histogram for counting the obtained frequency of each straight line until the value of the inclination θ set in the plurality of stages reaches 180 degrees. Is repeated, the frequencies of a plurality of types of straight lines passing through all the pixels in each area Ta are counted for each pixel.

When the counting of the frequency of the straight line with respect to each area Ta is completed, the maximum frequency is obtained by calculating the combination of the gradient θ and the distance ρ which has the maximum frequency from the value added to the two-dimensional histogram. One straight line L
x (see FIG. 6), and the straight line Lx is obtained as information obtained by linearly approximating a line segment L connecting the planted seedlings Tn adjacent to the unplanted area N on the imaging surface of the image sensor S1. .

Next, the straight line Lx on the image pickup plane is calculated by using the storage information of the shape and size of the image pickup visual field A of the image sensor S1 on the ground surface measured in advance and the image pickup plane through which the straight line Lx having the highest frequency passes. Are converted to information on a straight line L on the ground surface based on the pixel positions a, b, and c (see FIGS. 6 and 7). That is, as shown in FIG. 7, a straight line L on the ground surface set as a value of a slope ψ with respect to a running reference line La passing through the center of the imaging field of view A in the width direction in the front-rear direction and a position δ in the width direction. Will be converted into information.

In addition, the front and rear positions (y = 0 and y = 3) of the imaging field of view A in a direction intersecting with a straight line L corresponding to a line segment connecting the planted seedlings Tn adjacent to the unplanted side area N.
32), the lengths l ₀ and l ₃₂ of the two sides and the center of the screen (x = 1
6, the position of y = 16), the length l ₁₆ of the imaging field of view A in the width direction and the distance h between the front and rear sides are measured in advance and stored in the control device 12. deep. Then, the x-coordinate values of positions a and b (positions where y = 0, y = 32) where the straight line Lx on the imaging plane intersects the x-axis corresponding to the two sides at the front and rear positions of the imaging field of view A X ₀ , X ₃₂ and the value X ₁₆ of the x coordinate of the position c where the straight line Lx intersects the x axis passing through the center of the screen are obtained from the following equation (ii) obtained by modifying the above equation (i). . Xi = (ρ−Yi · sin θ) / cos θ (ii) Here, 0, 16, and 32 are substituted for Yi, respectively.

Then, based on the coordinate values on the x-axis obtained by the above equation (ii), the following formulas (iii) and (iv) are used to calculate the position δ in the lateral width direction with respect to the traveling reference line La.
And the slope ψ, and the obtained values of the position δ and the slope ψ
The position information is calculated as the position information of the straight line L corresponding to the planted seedling Tn on the ground surface.

[0029]

(Equation 1)

Accordingly, in the steering control for automatically moving the body V along the rows of the planted seedlings T arranged in the body traveling direction, the inclination of the straight line L with respect to the running reference line La and the width で in the width direction are determined. The steering operation is performed in a two-wheel steering system so that both the position δ and the position δ approach zero.
The steering control will be described. From the values of the inclination ψ of the straight line L with respect to the traveling reference line La and the position δ in the lateral width direction and the value of the current steering angle φ of the front wheel 1F, the following ( v) based on the equation, and sets the target steering angle θf of the front wheel 1F, and the current steering angle detected by the steering angle detecting potentiometer R ₁ for the front wheel φ is the target steering angle θf So that the hydraulic cylinder 7 for the front wheels is maintained within the dead zone.
The control valve 8F of F will be driven. θf = K ₁ δ + K ₂ ψ + K ₃ φ (v) K ₁ , K ₂ , and K ₃ are constants set in advance according to the steering characteristics.

When reaching the end of the work process, the planting operation by the seedling planting apparatus 2 is interrupted, and the seedling is turned toward the start of the next work process. In the turning operation, switching from the two-wheel steering type to the four-wheel steering type and maintaining the maximum turning angle for a set time, the vehicle is turned 180 degrees to the next work stroke side, and then the parallel operation is performed. By switching to the steering mode and maintaining the maximum turning angle for the set time, the position in the lateral direction of the machine body for the next work stroke is corrected, and the turn is completed. After the end of the turn, the steering is returned to the two-wheel steering mode, and the steering control in the next work process is restarted.

In the planting state detection process (FIG. 12), first, every time the planting phase detection sensor S4 is turned on, an image pickup process by the image sensor S3 is executed. still,
The timing at which the planting phase detection sensor S4 is turned on is
After the seedlings T are planted in the field by the planting claws 15, the machine body V travels a predetermined distance, and the planted seedlings T are positioned substantially at the center in the vertical direction of the frontmost screen area G in the imaging screen of the image sensor S3. (See FIG. 9A). Subsequent to the above-described imaging processing, the region extraction means 200 extracts each region Ta corresponding to each seedling T in the foreground screen region G. On this occasion,
A labeling process is performed on each of the extracted regions Ta (for example, under the 8-connection condition), and a number corresponding to the number of seedlings T in the foreground screen region G (the normal number in this example is six) is obtained. The regions Ta1 to Ta6 are determined (see FIG. 9B).

Next, the position of the center of gravity is determined as the representative point Q of each of the regions Ta1 to Ta6, and it is determined whether or not the number of the representative points Q is a normal number. If the number is not a regular number, a lack of stock has occurred, and the location of the lack of stock is determined. In FIG.
This shows that the fourth seedling T4 from the left of the screen is missing. If the number of representative points Q is a normal number, it is further determined whether or not the position of each representative point Q is at a predetermined position in the horizontal direction of the screen. Specifically, the lateral shift amounts Δd from the normal positions K1 to K6 respectively corresponding to the seedling planting locations T1 to T6 are detected, and when the lateral shift amounts Δd are equal to or greater than a predetermined value, it is determined that a lateral shift has occurred ( (See FIG. 10). In the figure, 3 from the left of the screen
This indicates that the lateral displacement has occurred in the third seedling T3. And
The discriminated missing position information and the laterally displaced strain position information (these correspond to the planting state discrimination information) are stored.

Next, it is checked whether or not the planting state discrimination information has been stored a predetermined number of times (for example, four times). If the planting state discrimination information has reached the predetermined number, the final judgment on the planting state is made from the discrimination information of all the times. I do. That is, as shown in FIG. 11, the foreground screen G (1) at the time of the current detection and the screens G (2), G
(3) and G (4) exist respectively, but in all of the discrimination information on each of these screens, it is determined that a stump of the seedling T4, which is a missing stock, has a seedling jam, and lateral displacement occurs. It is determined that the strip of the seedling T3 is a laterally shifted strip.
In addition, when a seedling jam or a lateral slip occurs, the control device 12 immediately stops running, and activates, for example, an alarm device to notify the worker of the occurrence of the abnormal planting condition.

[Another Embodiment] In the above embodiment, the image pickup means S
A region Ta corresponding to the crop T is extracted by subtracting the blue information B from the green information G and binarizing it based on the set threshold value by using a color image sensor as 3, for example. The specific configuration of the area extracting means 200 can be variously changed, for example, using all of the three primary color information R, G, and B to extract an area in which the ratio falls within the set ratio range as the area Ta. Also, the imaging means S3 is not limited to a color image sensor, and may be any as long as it can output image information that can identify a region corresponding to the crop T and a surrounding field region.

In the above embodiment, the image memory 21 as the information storage means is configured to store the discrimination information of the discrimination means 101 in a time-series manner. May be configured to be stored in chronological order, and the determining unit 101 may determine the planting state based on the entirety of the region extraction information at the plurality of times. Further, although the information storage means is constituted by the image memory 21, a dedicated memory different from the image memory 21 may be provided.

Further, in the above embodiment, the discrimination information of the discriminating means 101 (or the area extracting means 2) stored in chronological order.
Although the determination means 101 is configured to determine the planting state based on the above-described information at the present time, the determination means 101 determines the planting state based on only the above-described determination information at the present time. May be configured. As the imaging information in this case, only the information in the foreground screen area G may be used, but not only the information in the foreground screen area G but also the crop T
Is preferably used.

Further, in the above-described embodiment, the occurrence of seedling jam and lateral displacement for each row is exemplified as the planting state determination information determined by the determination means 101. However, other planting state determination information, such as the normal position of the plant, is used. It is also possible to determine the occurrence of a vertical displacement from the ground or the fall of a seedling.

In the above embodiment, the present invention is applied to an apparatus for planting seedlings T in a field while automatically operating a rice planting machine as a planting work machine. Can be applied to the planting work machine described above, and the specific configuration of each part at that time can be variously changed.

In the claims, reference numerals are provided for convenience of comparison with the drawings, but the present invention is not limited to the configuration shown in the attached drawings.

[0041]

[Brief description of the drawings]

FIG. 1 is a block diagram of a control configuration.

FIG. 2 is a schematic plan view of a rice transplanter.

FIG. 3 is a schematic side view of a rice transplanter.

FIG. 4 is a schematic rear side view of a rice transplanter.

FIG. 5 is an explanatory diagram of image processing.

FIG. 6 is an explanatory diagram of a Hough transform process.

FIG. 7 is an explanatory diagram showing a relationship between an aircraft traveling direction and an approximate straight line in an imaging field of view.

FIG. 8 is a flowchart of a boundary detection process.

FIG. 9 is an explanatory diagram of image processing.

FIG. 10 is an explanatory diagram of a planting state detection process.

FIG. 11 is an explanatory diagram of a planting state detection process.

FIG. 12 is a flowchart of a planting state detection process.

[Explanation of symbols]

 V body T crop S3 imaging means 200 area extraction means 101 discrimination means 21 information storage means

──────────────────────────────────────────────────続 き Continuation of the front page (51) Int.Cl. ⁷ identification code FI G05D 1/02 G05D 1/02 NG06T 1/00 G06F 15/62 380 (58) Fields surveyed (Int.Cl. ⁷ , DB A01C 11/02 G05D 1/02 G06T 1/00

Claims (2)

(57) [Claims] 1. In a field scene on the rear side in the body traveling direction,
An image pickup means (S3) for picking up an image of a predetermined area including a crop planting location where the crop (T) is to be planted as the machine body (V) advances, and the crop based on image pickup information of the image pickup means (S3). An area extracting means (200) for extracting an area corresponding to (T), and a judging means (101) for judging a planting state of the crop (T) based on information of the area extracting means (200) are provided. Planting condition detection device for crop planting machines. 2. An information storage means (21) for storing information of said discriminating means (101) in a time-series manner, wherein said discriminating means (101) is said information storing means (21).
2. The planting state detecting device for a crop planting machine according to claim 1, wherein the planting state is determined based on the stored information.