JPH0437443B2 - - Google Patents

Description

[Detailed Description of the Invention] [Industrial Application Field] The present invention captures an image of the working area condition in a predetermined range on the forward side in the direction of movement of the aircraft, and converts the captured image information into edge image information based on changes in brightness. , binarize the edge image information based on the set threshold, and
The present invention relates to a boundary detection method for an automatic traveling work vehicle that detects a boundary between an untreated work area and a processed work area by subjecting digitized image information to Hough change processing.

[Conventional technology]

In the above-mentioned types of autonomous work vehicles, such as lawn mowers and harvesters, the outer periphery of the scheduled work area is artificially treated as a treated work area in advance, and the area is surrounded by this treated work area. The vehicle travels along the untreated working area inside the area, along the boundary between the untreated working area and the treated working area, and runs in one process from one end of the untreated working area to the other. After that, the robot is controlled to automatically travel in the direction of the next process while working on a predetermined range of work areas. During each stroke, travel is controlled based on the boundary between the untreated work area and the treated work area in the width direction of the aircraft and the positional relationship with the aircraft, and at the end of the journey, the boundary in the longitudinal direction of the aircraft is controlled. The direction to the next leg is controlled based on the positional relationship between the aircraft and the aircraft.

The applicant claims that when the work area is viewed diagonally from above, the untreated work area appears dark due to the presence of uncut grass, while the treated work area appears brighter due to the light reflected by the cut grass. Focusing on this phenomenon, we developed a boundary detection device that applies the above method as a boundary detection means that can detect the shape of the boundary between untreated and treated work areas without actually driving on it. I have suggested it. (Special application 1986-
(See No. 163729) [Problems to be solved by the invention] However, in the above boundary detection method,
Since captured image information is converted into an edge image and binarized based on the brightness difference, the obtained binarized image information contains information other than the boundary information to be detected, but cannot be distinguished from true boundary information. It contains difficult information. Furthermore, even if it is true boundary information, the image information is not necessarily obtained as a continuous line;
It becomes intermittent due to the influence of noise components in the image and locally existing changes in brightness and darkness.

In this way, as a means of combining discontinuous line segments into continuous line segments or erasing isolated line segments,
It is known that Hough transform processing is effective.

However, although the Hough transform processing described above is an effective means of approximating discontinuous line segment information to one continuous line segment information, the calculations required for this process convert image information in an orthogonal coordinate system to a polar coordinate system. It is necessary to perform the conversion into , which involves arithmetic processing with a heavy processing load. Furthermore, since the processing has to be performed over all pixels, the amount of information to be processed increases, making high-speed processing difficult.

The present invention has been made in view of the above-mentioned circumstances, and its purpose is to enable the above-mentioned Hough transform processing to be performed easily and at high speed.

[Means for solving problems]

The feature of the boundary detection method for an automatic driving work vehicle according to the present invention is that, when performing the Hough transform processing, Hough transform data calculated in advance corresponding to the coordinates of the edge image information is tabulated, and the values of the coordinates are converted into a table. The Hough transform data tabled in correspondence is read out, and the most frequently read out data is used as information corresponding to the boundary, and its functions and effects are as follows.

[Effect]

For example, by configuring both memories so that the address value of the image memory that stores edge image information matches the address value of the Hough transform data memory that stores pre-calculated Hough transform data as a table, At the same time as the information is read out, the corresponding Hough transform data is read out, its frequency is counted, and the Hough transform data with the highest frequency is used as detection boundary information. In other words,
By calculating the necessary Hough transform data in advance and creating a table, it is possible to read the image information and directly obtain the Hough transform results of the image information without performing complicated arithmetic processing.

〔Effect of the invention〕

Therefore, it is possible to easily and quickly determine the boundary using only a simple counting process that does not involve complicated arithmetic processing, such as reading out Hough transform data that has been tabulated in advance in accordance with the coordinate values of edge image information and counting the readout frequency. This allows you to obtain specific information.

〔Example〕

DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment in which the method of the present invention is applied to a lawn mowing vehicle as an automatically traveling vehicle will be described below with reference to the drawings.

As shown in FIGS. 7 and 8, the lawn mower 2 is vertically movably suspended in the middle of the machine V, which is configured to be able to steer both the front wheels 1F and the rear wheels 1R. A video camera 3 is provided as an image capturing means for capturing an image of the working area in a predetermined range ahead in the direction of travel, and the image information captured by the camera 3 is binarized by a boundary detection device A to be described later. Boundary between untreated work area B and treated work area C in each direction
The positional relationship of the aircraft V with respect to L ₁ and L ₂ is detected, and based on the detection results of the boundaries L ₁ and L ₂ by the boundary detection device A, the travel control device H causes the aircraft V to move along the boundary L ₁ . It is configured to perform a steering operation to automatically travel, and to perform a steering operation and a gear change operation to automatically travel to the next step after one stroke is completed.

The camera 3 is installed at the tip of a support frame 4 that extends toward the upper front of the aircraft V so as to take an image of a predetermined range of work area on the front side of the aircraft from diagonally upward to downward. In a state where V is along the boundary L ₁ in the width direction, this boundary L ₁ is located at the center of the imaging field of the camera 3 in the vertical direction.

When detecting the boundary L ₁ in the width direction of the aircraft V, the detection boundary L _{1 is determined with respect to a reference line L 0 that} vertically traverses the center of the imaging field of the camera 3 in the vertical direction, with the lower end of the imaging field of the camera 3 as a reference. The positional deviation β is taken as the positional deviation information in the width direction of the aircraft V, and the inclination α of the detection boundary L 1 with respect to the reference line L ₀ that vertically traverses the center of the imaging field of view of the camera 3 in the vertical direction is defined as the inclination α of the detection boundary L ₁ of the aircraft V.
Detected as direction deviation information.

On the other hand, when detecting the boundary L ₂ in the longitudinal direction of the aircraft V, the vertical center of the imaging field of the camera 3 is The deviation β with respect to the reference line L that runs vertically is the distance information between the aircraft V and the boundary _L2 , and the inclination α with respect to the reference line L that runs vertically across the center of the imaging field of the camera 3 in the vertical direction is:
Aircraft V with respect to boundary L ₂ in the longitudinal direction of the aircraft V
The slope is In addition, the boundary in the front and rear direction of this aircraft V
When detecting L ₂ , the value of the detected deviation (β) does not immediately become the position information of the end of the stroke, so the value of the detected deviation (β) and the camera 3
The distance from the lower end of the imaging field of view to the aircraft V is set in advance based on the attachment distance to the aircraft V and the angle of the imaging direction, and the actual distance between the boundary L ₂ and the aircraft V is calculated based on this set distance information. That will happen.

By the way, since the image captured by the camera 3 is binarized based on the brightness difference between adjacent pixels, it is necessary to remove the influence of local brightness changes and average it. A soft focus filter 8 is provided in front of the camera 3 to blur the captured image.

Furthermore, since the camera 3 looks down diagonally from above the aircraft V, its imaging field of view becomes a trapezoid that expands from the near side to the far side, and the further away the image becomes, the more detailed the image becomes. Therefore, if the image information S ₀ captured by the camera 3 is sampled as is, local changes may be emphasized and noise components may increase as the image information is closer to the front, so the entire captured image may not be uniformly averaged. As shown in FIG.

Further, as a means for eliminating the influence of large brightness changes such as shadows of untreated grass caused by natural light, a strobe device 9 is provided which emits light in synchronization with the image taking by the camera 3.

As shown in FIG. 9, the driving force from the engine E is transmitted to each of the front wheels 1F and rear wheels 1R via the transmission 4, and the shift position detected by the shift position detection potentiometer _R3 is The motor 5 is driven so that the vehicle travels at a predetermined speed so as to reach a predetermined position.

The front wheels 1F and the rear wheels 1R are configured to be power-steering operated by hydraulic cylinders 6F and 6R, respectively, and are controlled by steering angle detection potentiometers R ₁ and R ₂ that are linked to the steering operation of the wheels. an electromagnetic valve 7F that operates the hydraulic cylinders 6F and 6R so that the detected steering angle matches the target steering angle;
It is designed to drive 7R. When correcting the tilt α of the aircraft V in the longitudinal direction with respect to the boundary L ₁ in the aircraft longitudinal direction, the front wheels 1F and the rear wheels 1R are steered in relatively opposite directions in order to change the aircraft V direction. When performing turning steering and correcting the deviation β in the width direction of the aircraft V with respect to the boundary L ₁ , perform steering operation so that the front wheels 1F and rear wheels 1R face in the same direction in order to move the aircraft parallel without changing the direction of the V. By performing parallel steering, the base body V is efficiently controlled to follow the boundary _L1 .

The configuration and operation of the boundary detection device A will be described below.

As shown in FIG. 1, the boundary detection device A quantizes the image information _S0 captured by the camera 3 with an 8-bit resolution and converts it into a digital value.
The D conversion unit 10 transforms the digitized image information by 90 degrees in its x and y coordinates so that the number of constituent pixels is 32×32.
An image memory 11 that samples and stores image information as image information, and image information stored in this image memory 11.
An image serving as an input format switching means for switching which of image information S ₂ and image information S ₁ output from the A/D converter 10 (however, the number of constituent pixels is processed as 32×32) is to be processed. Switcher 1
2. The image signal output by this switch 12
A differentiation processing unit 20 that differentiates F ₁ based on the brightness change in the aircraft width direction (x coordinate direction) and converts it into edge image data F ₂ corresponding to the directionality and magnitude of the brightness change, and this differentiation processing The differential sign ( _positive ,
By comparing the edge image data _F2 on the sign side (positive and negative) with the set threshold value (Fref), binarization is performed based on the magnitude of the brightness difference, and image parts with large brightness changes are extracted2. Value processing unit 3
0, a Hough transform unit 40 that approximates the binarized image data F ₃ binarized by this binarization processing unit 30 as a continuous straight line and identifies boundaries L ₁ and L ₂ on the image; A boundary parameter calculation unit 50 that converts the detection boundaries L ₁ and L ₂ into the following equation () approximated as a straight line representing the positional relationship with the aircraft V based on the information from the unit 40, and control that controls the operation of each processing unit. 60, an ITV controller 13 that controls the light emission of the strobe 9 and imaging of the camera 3, and a CRT controller that controls the operation of the monitor television 15 that displays images processed by the differential processing section 20 and the binarization processing section 30. 14,
It is composed of each of the following.

y=ax+b...() However, a: Boundary _L1 with respect to the reference line _L0 in the image coordinate system,
Inclination of L ₂ , b: Boundary L ₁ with respect to reference line L ₀ in image coordinate system,
Lateral deviation of _L2 , x: coordinate value of a pixel in the width direction of the aircraft V y: coordinate value of a pixel in the longitudinal direction of the aircraft V.

The overall operation will be described below based on the flowchart shown in FIG.

That is, the entire image is averaged and captured by the blurring filter 8, and the captured image signal _S0 is A/D converted to produce a digital signal in the orthogonal coordinate system x, y consisting of 32 x 32 pixels per screen. Transformation image signal
Convert to _F1 .

Next, detect the boundary L ₁ in the width direction of the aircraft in the middle of the stroke, or detect the boundary L 1 in the longitudinal direction of the aircraft at the end of the stroke.
If _{the boundary L 2 at} the end of the stroke is to be detected, the image memory 11
By converting the address coordinates of image signal S ₁ by 90 degrees and giving the image information S ₂ stored by converting the coordinates of image signal S 1 by 90 degrees, when detecting the boundary L ₁ on the way, the original image signal S Image signal S ₁ in the same coordinate system as ₀
The differential processing unit 20 performs differential processing on the image data using the differential processing unit 20 , and only those whose differential sign direction is positive or negative are binarized in the binarization processing unit 30 .

This selection of whether the differential sign is positive or negative is to be binarized is made as explained below.
The setting is made based on whether the unprocessed work value B exists on the left or right side. That is, for example, if we take a case where the work direction is circular mowing, in which the work direction is to move counterclockwise toward the inner circumference while turning each side of the outer circumference of the work area by 90 degrees, in this case, the machine V will move to the right side. The vehicle is now running adjacent to the treated work area C. In other words, when the brightness change is scanned from the left side to the right side of the image, a large positive change from dark to bright will occur at the boundary _L1 , and when the change between adjacent pixels is differentiated, the sign of the differential value will be A positive (+) value corresponds to a certain directionality of the boundary L ₁ when looking at the brightness change from the untreated work area (B) side to the treated work area (C) side, and a negative (-) value This can be considered as noise and removed.

Next, this differentially processed image data is binarized, and based on the following formula (), it is converted from an orthogonal coordinate system to a polar coordinate system and subjected to Hough transform, and the same Hough transform is applied at each polar coordinate (ρ, θ). The frequency of binarized image data that takes data ρ is counted as a two-dimensional histogram, and its maximum value (Dmax) is determined.

ρ＝xcosθ＋ysinθ……() however, 0 degree≦θ≦180 degree x: Coordinate value of the pixel in the width direction of the aircraft V y: Coordinate value of a pixel in the longitudinal direction of the aircraft V It is.

Then, from the value of the Hough transform data ρ of the frequency D that is the maximum value (Dmax), the above equation (), which is a boundary parameter corresponding to one boundary information, is determined, and the slope as a coefficient of this equation () is determined. From a and the deviation b, in the travel control device H, the boundary
Convert the actual inclination α and deviation β in the width direction of the aircraft V with respect to L ₁ , and steer the front wheels 1F and rear wheels 1R so that these inclination α and deviation β become zero, respectively. It controls the vehicle to automatically travel along _L1 .

By the way, when detecting the boundary L ₁ in the width direction of the aircraft, the imaging information from the camera 3 is input as is to the differential processing unit 20 and processed, and when detecting the boundary L ₂ in the longitudinal direction of the aircraft, the original image The vertical direction y of the information is the horizontal direction x, and the horizontal direction x is the vertical direction y.
The coordinates x, y of the imaging information are converted by 90 degrees by being converted and stored in the image memory 11, and read out with the original coordinate system x, y unchanged, and then input to the differential processing section 20. Since the processing is performed, each processing section after the differential processing section 20 can be used as is to detect boundaries L ₁ and L ₂ in different directions with respect to the aircraft V.

When detecting the boundary L ₂ in the longitudinal direction of the aircraft, if the maximum value (Dmax) of the frequency matching the same Hough transform data ρ is not greater than the set threshold (Dref), the aircraft V is still at the end of the stroke. It can be determined that the area near the boundary _L2 has not been reached. On the other hand, if the maximum value (Dmax) is greater than or equal to the set threshold (Dref), based on the slope a and deviation b of the equation () and the distance between the lower end of the imaging field of view and the front end of the aircraft V,
By determining the distance from the aircraft V to the end-of-stroke boundary _L2 , the turning point can be determined accurately.

Hereinafter, the operation of each part will be explained based on the drawings shown in FIGS. 3 to 6.

First, the configuration and operation of the control section 60 will be explained based on the block diagram shown in FIG.

The entire operation of the control section 60 is performed in synchronization with the clock signal CL generated by the clock generator 61a, and the CPU 1 (6) serves as a control processor that controls the operation of each section.
2a) and CPU2 as a processor for numerical calculations
Two processors (62b) are provided to enable high-speed processing. The calculation results of each control signal and data are sent to the bus buffer 63.
The information is exchanged between each processing unit via the . Similarly, data exchange with the travel control device H is performed online via an SIO port 64a for serial interface and a communication interface 64b connected to the SIO port 64a. , the boundary detection device A can also be activated offline via the online switch interface 64c and the parallel interface PIO port 64d.

In addition, CPU1 (62a) which is the processor
The input/output controller 65 controls the data transfer and operation between the CPU 2 (62b) and the bus buffer 63, and also controls the data transfer from the ports 64a and 64d. The CTC controller 6 is controlled by the control signal from the output controller 65.
6. In FIG. 3, 61b is a clock signal generator that generates operating clock signals for the SIO port 64a and the CTC controller 66, and ROM 67 is a memory and RAM in which the operating program of the boundary detection device A is stored.
68 is a memory used for temporarily storing each data or as a buffer for calculation data, and LUT 69 is Hough transform data ρ in a polar coordinate system from the orthogonal coordinate values x, y of each pixel of image information used in the Hough transform unit 15.
This is a memory in which Hough transform data ρ can be directly obtained from the calculation data for high-speed conversion, that is, the values of the coordinates x and y.

As shown in FIG. 4, the LUT 69 calculates the angle (θ) of the polar coordinates (ρ, θ) (however, 0 Hough transform data ρ for each angle is calculated in advance by dividing the angle (degree ≦ θ ≦ 180 degrees) into 256 equal parts, and the 256 Hough transform data ρ for each coordinate value x, y, that is, each pixel, is stored in ROM. As shown in FIG. 5, address information for reading out the stored Hough transform data ρ based on the given coordinate values x, y of each pixel is stored. A pair of x registers 69a, 6 to generate
9b and y register 69c, a θ counter 69d that generates 256 θ address information for each pixel, and the Hough transform data based on the address information generated by each of the registers 69a, 69b, 69c and the θ counter 69d. The address decoder 69e generates real address information of a Hough transform data memory 69d storing ρ.
Then, for each given pixel coordinate value x, y,
The configuration is such that 256 pieces of Hough transform data ρ are directly output to the Hough transform unit 40 as Hough transform results together with information on the angle (θ) generated by the θ counter 69d.

Next, the operation of the differential processing section 20 will be briefly explained.

The video signal _S0 , which is the original image information from the camera 3, is transmitted directly through the A/D converter 10 consisting of a video amplifier 10a and an A/D converter 10b, or after 90 degree coordinate transformation, the video signal S0 is converted into the image. As image information stored in the memory 11, 32 x 32 pixels, 8 pixels per pixel, are input via the input format switch 12.
Digital image data F1 expressed in bit length
is input to this differential processing section 20 as follows.

Then, one raster (all 32 pixels of all x coordinates for one y coordinate) of image data F ₁ is sequentially obtained from left to right with the upper left of the imaging field of camera 3 as the origin.
While importing 32 rasters (32 pixels worth of y coordinates), the process of calculating the differential value for each pixel is performed. This differential processing is performed in the left and right direction (x-axis direction) for each pixel based on the brightness of the pixel.
By calculating the differential value of , the direction and magnitude of the change in brightness of all image information is determined.
This makes it possible to extract the edges of the change from dark to bright in the image.

Next, based on the block diagram shown in FIG. 6, a binarization processing unit 30 that binarizes the image information converted into 8-bit/pixel edge image data _F2 by the differential processing unit 20, and a Hough transform part 40, and
The configuration and operation of the boundary parameter calculation section 50 will be explained.

That is, the entire operation is controlled by the input/output controller 42 so that the entire operation is performed in synchronization with the operation of the differential processing section 20 based on the control signal from the control section 60 inputted via the bus buffer 41. It is configured so that

The edge image data _F2 is input to the comparator 31 of the binarization processing unit 30 via the data buffer 43a, and is compared with a set threshold value (Fref) input via the buffer 32, and portions with large brightness changes are identified. Binarized. This binary image data
F ₃ is input to the comparator 51 of the boundary parameter calculation section 50 via a data buffer 43b connected to a data bus (BUS ₃ ), and is also stored in the binarized image data memory 33a.

The Hough transform unit 40 uses a frequency counter 44 to count the frequency D of angles (θ) at which the values of the Hough transform data ρ given from the LUT 69 via the data buffer 43c are the same value, and converts the result into a two-dimensional It is stored in the histogram frequency memory 45.

The frequency D counted by the frequency counter 44
is added to the contents of the same address position in the frequency memory 45 via the data buffer 43d, and a boundary parameter calculation unit 50 specifies the boundary by detecting its maximum value (Dmax).
The value is transferred to the maximum value register 52a, and compared with the value stored in the maximum value register 52a by the comparator 51, and the larger frequency is re-stored as the new maximum value (Dmax). The address of the frequency memory 45 for the frequency D that is the maximum value (Dmax) is stored in the maximum value address register 52.
b at the same time, and after Hough transformation of all pixels (32×32) is completed, the above formula () is calculated from the Hough transformation data ρ and angle (θ) that have reached the maximum value (Dmax).
The coefficients (a, b) of or,
In FIG. 6, 33b is the digitized image data.
33c is an image memory that stores edge image data _F2 , and 33c is an image memory that stores edge image data _F2 ;
Each processing status can be displayed on the monitor television 15 via the RAM 34 and the CRT controller 14.

Then, the coefficients (a, b) of the obtained formula ()
is sent to the control section 60 via the input/output controller 42 and bus buffer 41, and
It is used as data for determining the presence or absence of L ₁ and L ₂ and for subsequent processing.

[Another example]

In the above embodiment, the boundary L ₂ in the longitudinal direction of the aircraft V
The differential processing unit 20
The coordinates of the input image information are converted by 90 degrees and 90
However, each time one screen's worth of image information is processed, the coordinates of the input image are converted 90 degrees, and the boundary _L1 in the width direction of the aircraft V is detected. Detection of the boundary _L2 in the V front and rear directions may be repeated alternately. In that case, if you alternately input the coordinates transformed and the coordinates not transformed for each field of the image captured by the camera 3, the necessary information can be obtained for each frame at the same time, and the imaging processing for one screen can be performed. Image information for detecting boundaries L ₁ and L ₂ in different directions between the two can be obtained.

Alternatively, the aircraft V is provided with means for detecting the travel distance information, and based on the detected travel distance information and the scheduled travel distance information for one stroke, the aircraft V is adjusted in the longitudinal direction depending on whether or not it has reached the vicinity of the end of the stroke. boundary of
It is also possible to switch to _L2 detection processing.

In addition, in the above embodiment, the formula for Hough transform processing ()
In the above, the angle change is processed between 0 degrees ≦ θ ≦ 180 degrees, but due to the positional relationship between the boundaries L ₁ and L ₂ and the aircraft V, the directions of the boundaries L ₁ and L ₂ are significantly different from the traveling direction of the aircraft V. Therefore, by limiting the angle range for Hough transform to a range narrower than 0 degrees to 180 degrees, high accuracy can be achieved.
Alternatively, by processing only a narrow specific range, the processing speed can be further improved or the influence of noise can be effectively removed.

In addition, in the above embodiment, the captured image information is 32×32
It was sampled into pixels, but if necessary, it can be made even more finely.
Alternatively, it may be roughly sampled.

In addition, to determine the positional relationship between the boundaries L ₁ and L ₂ and the aircraft V, in addition to using the reference line L ₀ that vertically traverses the imaging field of view as exemplified in the above embodiment, the lower and upper ends of the imaging field of view are used as a reference. Any reference may be used, such as the area, the center, or the coordinate origin position of the image information.

[Brief explanation of drawings]

The drawings show an embodiment of the boundary detection device for an automatic traveling work vehicle according to the present invention, and FIG. 1 is a block diagram showing the configuration of the boundary detection device, FIG. 2 is a flowchart showing its overall operation, and FIG. FIG. 3 is a block diagram showing the configuration of the control section, FIG. 4 is an explanatory diagram of a table storing data for Hough transformation, FIG. 5 is a block diagram showing the configuration of a data memory for Hough transformation, and FIG. A block diagram showing the configuration of the conversion section, FIG. 7 is a schematic plan view of the lawn mowing vehicle, and FIG. 8 is a side view thereof.
FIG. 9 is a block diagram showing the overall configuration of the control system for the lawn mowing vehicle. S ₀ ... Captured image information, F ₂ ... Edge image information,
_F3 ...Binarized image information, B...Unprocessed work area,
C...processed work area, L...boundary, x, y...coordinates of edge image information, ρ...data for Hough transformation,
D... Frequency.

Claims (1)

[Claims] 1 Capture an image of the working area condition in a predetermined range in front of the aircraft in the forward direction, convert the captured image information S ₀ to edge image information F ₂ based on changes in brightness, and convert the edge image information F ₂ based on a set threshold value. By binarizing the binarized image information _F3 and subjecting it to Hough transform processing, the boundary L between the unprocessed work area B and the processed work area C is
This is a boundary detection method for an autonomous driving vehicle that detects the edge image information F2, and in order to perform the Hough transform processing, Hough transform data ρ calculated in advance corresponding to the coordinates x and y of the edge image information _F2 is tabulated. , reads the tabulated Hough transform data ρ corresponding to the values of the coordinates x, y, and calculates the read frequency D.
A boundary detection method for an automatic traveling work vehicle in which the highest value is determined as information corresponding to the boundary L.