JP2740268B2 - Image information binarization processing device in steering control of traveling work machine - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION [Industrial Application Field] The present invention relates to a method of cutting and threshing by running a traveling machine along a row of rice, wheat, soybean, etc., planted in a row in a field. In a harvester such as a so-called general-purpose combine or a normal combine, or a traveling work vehicle such as a lawn mower, binarization of color image information obtained by imaging a state of a work place such as mowing is performed. The present invention relates to a color image information binarization processing device for using data for automatic steering control.

[Conventional technology]

Detects the boundary between uncut and already-cut areas when the combine is automatically steered so that the combine is moved forward along the row of rice planted in the field and the row of rice is cut in multiple rows. As a technology to do this, the applicant has
No. 63-263794, a color image pickup means is mounted on a combine, and the color image pickup means uses the three primary color components in a sampling area which is an uncut area to distinguish between an uncut area and a previously cut area. While a predetermined binarized threshold condition expression is created, an image is taken over both the uncut area and the already-cut area in the forward direction of the combine during traveling, and the three primary color components of the color image data are extracted. Proposed a technique of determining whether or not the binarized threshold condition expression is satisfied, and detecting a boundary between an uncut area and a previously-cut area.

[Problems to be solved by the invention]

The binarized threshold condition expression in the prior art is set as three inequalities each including a predetermined deviation for each of the three color components in the sampling area of the uncut area.

In other words, when the levels of the three color components at the location to be inspected are within the ranges of the three inequalities (when the inequalities are satisfied), it is determined that the ground has not been cut.
When the inequality was not satisfied, it was determined that the land was already cut.

If the location to be identified is completely different in color from the other locations, for example, green or yellow is strong in an uncut area, but the color of the ground is different (black or gray) in an already-cut area. In the case where it is easy to make a distinction, it is possible to discriminate between the two even with the binarized threshold condition formula obtained independently for each color component as described above.

However, when performing harvesting work in the combine harvester, such as cutting chopped cereal stalks (straw straw) and releasing them to the already-cut area, the cultivation of planted cereal culms such as rice in the uncut area is not possible. Since the color and the color of the straw after cutting are of the same color system, the value of the color component of the uncut area and the value of the color component of the already cut area are approximated by the color image information obtained by the color imaging means. Therefore, in the above-described binarized threshold condition expression obtained independently for each color component,
There is a problem that it is difficult to distinguish the two regions.

An object of the present invention is to solve this problem.

[Means for solving the problem]

In order to achieve this object, the present invention provides a color image pickup device mounted on a traveling work machine, which captures an image of a work surface between an uncut area and an already-cut area, such as a field, and reads the uncut area and the already-cut area from the color image information. An image processing apparatus for performing a binarization process for distinguishing between a reaping area and a three primary color signal of a blue component signal value (B), a red component signal value (R), and a green component signal value (G) from color image information. Signal value extracting means for extracting a value, and [B / (R + G)], [R / (B +
G)], calculating means for calculating three color component ratios of [G / (B + R)], and calculating the three color component ratios in the sampling area of the uncut area and the three color component ratios in the work surface, respectively. And a determining means for determining a binarization threshold value for comparing and judging the uncut area and the already-cut area.

〔Example〕

Next, a description will be given of an embodiment in which the present invention is applied to a general-purpose combine. A combine 1 that is a traveling work machine includes a pair of right and left crawler-type traveling devices 3, 3 on a lower surface of a traveling machine body 2,
A steering unit 4 with a seat is disposed at a front right portion of the traveling body 2 in the forward direction of the traveling body 2, and an engine (not shown) and a tank 5 for accumulating grains are provided at the rear thereof. On the left side, a threshing unit 6 with a built-in front and rear longitudinal handling cylinder 6a and the like in which a double-pitch screw plate and an appropriate number of teeth are implanted on the outer peripheral surface, a receiving net 8 below, a sheave, etc. The apparatus includes an oscillating sorting device 9 and a wind sorting device based on the wind of Karamino Juan 10.

On the other hand, a rectangular tubular feeder house 11 communicating with the entire opening of the threshing unit 6 is mounted on the front surface of the traveling body 2 via a hydraulic cylinder 12 for elevation so as to be able to move up and down with respect to the traveling body. In the interior of 11, a chain conveyor 13 in which left and right transport plates are attached to a pair of left and right chains at appropriate intervals is disposed, and the chain conveyor 13 removes the harvested grain culm from the cutting unit 15 described below to the threshing unit. 6.

The reference numeral 7a designates a grain from a second gutter (not shown) for the threshing unit 6
Reference numeral 7b denotes a discharge cylinder for discharging the grains in the tank 5 to the outside.

The reaping section 15 includes a bucket-shaped platform 17 extending left and right over the entire width of the traveling body 2,
17 comprises a reel 19 with a tine bar 18 provided over the entire width of the traveling body 2, and has a clipper-shaped cutting blade 20 which is also left and right on the lower surface side of the platform 17, and an arrow above the bottom plate of the platform 17. The auger 22 has a horizontally long scraping auger 22 that rotates in the direction, and drives the reel 19 and the auger 22 to rotate and drives the cutting blade 20 via a transmission unit.

Reference numeral 23 denotes a weeding body projecting forward from both the left and right ends of the reaper 15, and the reel 19 is configured to be capable of adjusting the forward and backward movement and the up and down swing according to the falling state of the grain culm and the like. I have.

Reference numeral 24 denotes a grain culm of rice, which is an object to be detected, which is planted vertically and horizontally at appropriate intervals in a field, as shown in FIG. The combine 1 is moved forward to mow them.

FIG. 3 shows a block diagram of the control means of the present invention, wherein reference numeral 25 denotes a central control unit (CPU), 26 denotes a read-only memory (ROM), 27 denotes a readable / writable memory (RAM), and 28 denotes a harvester 1 This is an odometer that measures the mileage.

Reference numeral 31 denotes an azimuth setting device which can manually set the azimuth (target azimuth) read from the azimuth detector 30 such as a geomagnetic sensor when the harvester 1 is oriented in advance in the traveling direction. The orientation can be modified.

The comparator 32 has a target azimuth set by the azimuth detector 30 and an azimuth setting device in a state where the harvester is running automatically.
This is to compare the difference from the actual azimuth detected at 31.

Numeral 33 is a non-contact type cereal stalk detection sensor using ultrasonic waves or the like, and the cereal stalk detection sensor 33 composed of a pair of a transmitter and a receiver is provided on the inside of the weeding body 23 in the cutting unit 5. Of the planted cereal stems that are mounted sideways and are located within the left and right widths of the platform 17 in the mowing part 5,
The receiver receives the reflected sound of the ultrasonic wave transmitted toward the grain culm closest to 23, and depending on the time from transmission to reception, the traveling direction of the traveling aircraft from the side of the weeding body 23 Detects the detection distance to the grain stem located in the horizontal direction at a right angle,
In addition, the presence or absence of a grain culm is determined by utilizing the fact that ultrasonic waves are blocked by the grain culm.

Reference numerals 34 and 35 denote right and left traveling devices 3 and 3 according to an output signal (steering command) from the central control device 25.
An electromagnetic solenoid such as a hydraulic switching valve that operates a steering clutch that turns off power transmission to the steering clutch. The steering clutch can also be operated by a manual lever (not shown).

The color imaging means 36 for obtaining color information is a two-dimensional plane having an image plane (imaging plane) extending vertically and horizontally like an xy plane when detecting an object. For example, in the case of incorporating a two-dimensional MOS image sensor or two-dimensional CCD image sensor, the image formed through the lens is a two-dimensional array on the image plane. The optical information of the imaging screen can be output as an electric signal by being sensed by an element (photoelectric element).

The color imaging means 36 includes a red component (R) and a green-white component according to an RGB color system [a color light of red (R), green (G), and blue (B) is used as primary color light and white is obtained by adding light]. (G),
The features of the field scene can be extracted from the signal of each color component with the blue component (B).

Reference numeral 37 denotes an A / D converter that converts an output signal from the color imaging unit 36 from an analog signal to a digital signal.
Reference numeral 8 denotes an image memory for storing image information for each captured image. The image memory 38 stores color information for, for example, 64 × 64 dots (vertical × horizontal) in a red component (R) and a green component (G). , As the level of the color signal of the blue component (B)
It is extracted and stored in 64 gradations (0 to 63).

The binarization processing device 39 is a binarization conditional expression creating means for determining a binarization threshold based on the sample data composed of the three color components obtained from the sample image information. Determining means for executing an arithmetic operation and determining whether or not the detected color component data composed of the three color components obtained from the detected image information at the time of automatic driving later satisfies the predetermined binarization condition expression; It is.

Reference numeral 40 denotes a read-only memory (ROM) for the sub-control device 39, and 41 denotes a readable / writable memory (RAM).

Next, the binarization processing according to this configuration will be described with reference to the flowchart shown in FIG. 6. In the teaching running, after the start and the initial value setting of step S, as shown in FIG. When the color imaging unit 36 takes an image of an appropriate range extending between the scattered work area (K1) on the right side and the unworked area (M1) on the left in the traveling direction,
As shown in the figure, on the imaging screen 43, an image (K) of a work place is shown in a substantially half area on the right side, and an image (M) of an unworked place is shown in a half area on the left side.

If an appropriate range surrounded by a dotted line is adopted as a sample area (SAMP) in a screen approximately half left on the imaging screen 43,
In this part, sample image information is obtained as color information of only the unworked land (for example, only the area where the cereal stem is planted).

In step S2, all or a plurality of pixels in the sample area (SAMP) are sampled, and the sample data is stored in the image memory 38 via the A / D converter 37.

That is, in step S2, for each of the sample areas (SAMP), the sample data is decomposed into three color components of a red component (R), a green component (G), and a blue component (B), and the levels R, G, B are obtained. Is read and stored.

In step S3, a component ratio is calculated for each of the three color components as a ratio of each color component to the other two color components.

here, It is.

Next, from the sample data obtained in step S3, the average value and the standard deviation of the respective color component ratios of the three color components in the sample area are calculated.

Here, rav is the average value of the color component ratio Rs of the red component, gav is the average value of the color component ratio Gs of the green component, bav is the average value of the color component ratio Bs of the blue component, and rv, gv , bv
Similarly, the standard deviation of the color formation ratio of each color component for each sample area, (N indicates the number of samples).

Further, in step S5, an upper limit value and a lower limit value for the binarized threshold condition expression are calculated in advance.

Here, the upper limit Rsh = rav + rv for the red component, the lower limit Rsl = rav-rv, and the upper limit Gsh = gav + g for the green component.
v, lower limit Gsl = gav−gv, upper limit Bsh = blue component
bav + bv, lower limit value Bsl = bav−bv.

In addition, at a plurality of sampling points as appropriate in the field,
The above-described sampling may be performed to calculate a binarization conditional expression from an average value of the sample data, or one field may be represented by one binarization conditional expression. good.

Next, unmanned automatic running is started. In step S6, during the automatic running, at appropriate time intervals (measured by a clock built into the central control device 25) or at appropriate mileage (measured by the odometer 28), The color imaging means 36 images an appropriate range over both the already-cutted and un-cutted fields in the field,
The detected image information is stored in the image memory 38 via the A / D converter 37.

In this case, the levels of the red component (Rx), the green component (Gx), and the blue component (Bx) are set for each of a plurality of predetermined sample points in the screen in the one piece of detected image information (one screen). Detect and read.

In step S7, the color component ratio of each color component is calculated for each sample point.

Here, Rxs = [Rx / (Gx + Bx)], Gxs = [Gx / (Rx + Bx)], and Bxs = [Bx / (Gx + Rx)].

In step S8, it is determined whether or not the corresponding binarization condition expression (A) is satisfied for each sample point.

Here, as an example of the binarization conditional expression (A), Rsh (= rav + rv)>Rxs> Rsl (= rav-rv)... Gsh (= gav + gv)>Gxs> Gsl (= gav-gv) Bsh (= Bav + bv)>Bxs> Bsl (= bav−bv) When the following three inequalities are simultaneously satisfied (yes), the binary level val =
1 (step S9), and when any one of the inequalities is not satisfied (no), the binary level Val =
It is set to 0 (step S10).

It goes without saying that the binary levels Val may be reversed from "1" and "0".

As another embodiment of the conditional expression for determining the binarization threshold value, a color component ratio is obtained in each of a sampling area in an uncut area and a sampling area in a already-cut area as in (1) above. The average value and the standard deviation are calculated from the color component ratios in. Then, the upper limit and the lower limit of the binarization threshold value in the uncut area are set as follows for each color component.

For red component: upper limit Rshm (= rav + rv), lower limit Rslm (= rav-rv) For green component: upper limit Gshm (= gav + gv), lower limit Gslm (= gav-gv) For blue component: upper limit Bshm (= bav + bv), lower limit Bslm (= bav−bv), and similarly, the upper and lower limits of the binarization threshold in the already cut area are as follows: for the red component: upper limit Rshk, lower limit Rslk For the green component: upper limit Gshk, lower limit Gslk : When the upper limit Bshk and the lower limit Bslk are set, for each color component, the range of the value between the upper limit and the lower limit of the binarization threshold in the uncut area, the upper limit and the lower limit of the binarization threshold in the already cut value, and Are compared with each other, and a color component whose ranges do not overlap each other is selected, and a conditional expression (A1) for determining a binarization threshold value for this color component is set as follows.

For example, assuming that the red component can be selected, Rshm>
"1" when Rxs> Rslm and "0" otherwise
Or “0” when Rshk>Rxs> Rslk, and “1” otherwise.

The boundary line obtained by executing the binarization process as described above is obtained as a virtual straight line by Hough transform or the like, and the electromagnetic solenoids 34 and 35 are operated so that the traveling body 1 advances along the virtual straight line. The cutting and threshing operation can be performed by running the combine along the boundary between the uncut area and the already-cut area.

Note that sampling of sampling data is not limited to one time only at the beginning, and a binarization conditional expression for the next run is created based on the data of the sampling area in the uncut area for each image capture during automatic running. Then, the binarization condition expression may be sequentially updated.

[Functions and Effects of the Invention]

As in the difference between the uncut area and the already-cut area, when the signal levels of the similar colors are similar to each other, the signal level of each color component of the three primary colors is determined solely as in the related art. In this case, the difference of each color component between the uncut area and the already-cut area hardly appears.

In addition, when the sum of the three primary color components is taken as the denominator and the ratio of each color component to that is obtained, the brightness is simply ignored, and the difference between the color components between the uncut and the already cut is small. Sometimes, the difference cannot be emphasized and discriminated.

However, in the present invention, the color component ratio of the red component Rs = [R /
(B + G)], a green component color component ratio Gs = [G / (B +
R)], the ratio of one color component to the other two color components, such as the color component ratio Bs = [B / (R + G)] of the blue component, is calculated based on the ratio. Therefore, even if the difference of each color component is small between the uncut area and the already-cut area, the result can be obtained in a form that emphasizes the difference, so that the two areas of the same color can be distinguished. It becomes easier.

Further, as means for determining a threshold value in the binarization process, a traveling body travels in advance in a field, samples image information of an unworked place, and uses a color component ratio of the three color components as a basis. Since the binarization condition expression is created, the threshold value can be set to an optimum value according to the imaging situation.
It is possible to clearly distinguish the boundary between the unworked land and the already worked land.

[Brief description of the drawings]

1 shows an embodiment of the present invention, FIG. 1 is a plan view of a combine, FIG. 2 is a side view, FIG. 3 is a block diagram of the apparatus of the present invention, FIG. FIG. 6 is an explanatory diagram of an image pickup screen, and FIG. 6 is a flowchart. 1 ... combine, 3,3 ... traveling device, 15 ... reaper, 2
5 Central control unit 36 Color imaging means 37 A / D
Converter 38 image memory 39 binarization processing device.

Claims (1)

(57) [Claims] An image of a work surface between an uncut area and an already-cut area such as a field is imaged by a color image pickup means mounted on a traveling work machine, and the uncut area and the already-cut area are distinguished from the color image information. Signal value for extracting three primary color signal values of a blue component signal value (B), a red component signal value (R), and a green component signal value (G) from the color image information Extracting means, from each of the signal values,
[B / (R + G)], [R / (B + G)], [G / (B +
R)], and the three color component ratios in the sampling area of the uncut area and the three color component ratios in the work plane are determined by comparing the three color component ratios in the sampling area of the uncut area. A binarization processing device for image information in steering control of a traveling work machine, comprising: a determination unit that determines a binarization threshold value that is distinguished from a previously cut area.