JPH0575335B2 - - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION [Field of Industrial Application] The present invention relates to an imaging type boundary detection device that detects a straight line corresponding to the boundary between an untreated work site and a treated work site.

[Conventional technology]

This type of imaging-type boundary detection device described above takes advantage of the fact that the brightness of an untreated work area and a treated work area appears to be different, and processes the captured image information to detect changes in brightness. A straight line passing through a large portion is detected as information corresponding to the boundary between the untreated work area and the treated work area.

Conventionally, the work area is imaged in two-dimensional directions, and from the imaged information, the differential value of the brightness change in the direction intersecting the boundary is calculated for each quantized pixel arranged in the two-dimensional direction. Based on the differential value, pixels at locations where brightness changes are large are extracted by binarization processing, and then, using Hough transform processing, each extracted pixel is passed through and set. A large number of straight lines with different directions at different angles were found, and of all the straight lines found, the one with the highest frequency was detected as information indicating the straight line corresponding to the boundary.

[Problem that the invention seeks to solve]

In the above conventional configuration, a straight line passing through a portion with a large brightness change is detected as a straight line corresponding to the boundary based on the imaging information. If you enter the imaging field of view while facing in the same direction as the boundary between the treatment work area and the treated work area, the brightness change will be large in the shadow area, so a straight line that passes through the imaged shadow area may be mistakenly detected as a straight line corresponding to the boundary.

Therefore, it is conceivable to use a strobe light emitting device or the like to eliminate the shadows, but since the imaging means images the work area in two dimensions, even if one with a large amount of light is used, It is difficult to completely eliminate shadows in areas where the imaging distance is far.

The present invention has been made in view of the above-mentioned circumstances, and an object of the present invention is to accurately draw a straight line corresponding to a boundary without being affected by the shadow, even if a shadow with a large change in brightness in the image information is captured. The point is to make it possible to detect it.

[Means for solving problems]

The characteristic configuration of the imaging type boundary detection device according to the present invention includes an imaging means for imaging the work area in two-dimensional directions, and a method for detecting brightness based on the image taken by the imaging means in a direction intersecting the boundary. a differentiation means for obtaining the differential value of each quantized pixel arranged in a two-dimensional direction; and 2. extracting a pixel for which the obtained differential value is large.
a pixel extracting means for searching in a direction intersecting the boundary and from dark to bright, and extracting the first existing pixel among the pixels extracted by the binarizing means; and a straight line calculating means for calculating a straight line passing through the pixels extracted by the pixel extracting means, and the functions and effects thereof are as follows.

[Effect]

If a shadow appears in a bright part of the captured image, the brightness change between the bright part and the shadow part will be large and cannot be removed by binarization, but if a shadow appears in a dark part, Since the brightness change is small, it can be automatically removed by the binarization means. In addition, since the parts with large brightness changes correspond to the boundaries of the work area, the pixels extracted by the binarization means should be searched from dark to bright and the first existing pixel should be extracted. For example, the extracted pixel is a pixel corresponding to the boundary part,
By excluding pixels after the first pixel from processing, shadows in bright areas can be automatically removed.

In other words, each quantized pixel of image information captured in a two-dimensional direction is arranged in a two-dimensional direction based on the brightness change in the direction that intersects the boundary between the unprocessed work area and the processed work area. By calculating the differential value of each quantized pixel and extracting pixels with large differential values using a binarization means, areas with large brightness changes are extracted, and unprocessed work areas and processed work areas are extracted. The first pixel among the pixels extracted by the binarization means is extracted by searching in the direction intersecting the boundary between the two and from dark to bright, and the extracted pixel is By finding the straight line that passes through it, it is possible to detect the straight line that corresponds to the boundary without being affected by shadows.

〔Effect of the invention〕

Therefore, the first existing pixel extracted by the binarization means is searched in the direction intersecting the boundary between the unprocessed work area and the processed work area and from dark to bright. Through the simple process of extracting pixels, it has become possible to accurately detect straight lines corresponding to boundaries without being affected by shadows.

〔Example〕

Embodiments of the present invention will be described below based on the drawings.

As shown in Fig. 7, a pair of left and right front and rear wheels 1, 2
However, each of them is provided so that the steering can be operated freely, and a lawn mowing device 3 is suspended on the lower abdomen of the vehicle body V so as to be movable up and down, thereby forming a work vehicle used for cutting grass, weeds, etc. There is.

An image sensor _S1 serving as an imaging means for capturing an image of the work area in two-dimensional directions is provided so as to image a predetermined range of the work area on the front side with respect to the vehicle body V.

To explain the imaging field of view of the image sensor _S1 , the vehicle body V is in an appropriate state with respect to the boundary L between the unmown land B as an untreated work land and the mowed land C as a treated work land. The boundary L is positioned at the center of the field of view in the width direction.

To explain the structure of the vehicle body V, as shown in FIG. 6, there are hydraulic cylinders 4 and 5 for steering that operate the front and rear wheels 1 and 2 separately, and
Control valves 6 and 7 are provided therefor, and a hydraulic continuously variable transmission 8, which can freely switch between forward and backward directions and can freely change speeds in both forward and backward directions, is interlocked with the engine E.
A speed change motor 9 is operatively connected to a speed change arm 10 of the speed change device 8.

Further, in order to detect the traveling distance of the vehicle body V,
A rotation speed sensor S ₂ that outputs a set number of pulse signals per unit rotation speed is provided so as to be rotationally driven by the output of the transmission 8 .

A steering position detection potentiometer that detects the steering position of each of the front and rear wheels 1 and 2.
R ₁ , R ₂ , and a shift position detection potentiometer R ₃ that detects the operating state of the transmission 8 are provided, and the detection information thereof and the image sensor S ₁ are provided.
Based on information on _a straight line corresponding to the boundary L determined based on the imaging information of A control device 11 using a microcomputer is provided to control the vehicle to automatically travel along the same route.

In the figure, H is a steering handle for boarding operation, R ₀ is a potentiometer for detecting its operation position, and 12 is a speed change pedal for boarding operation.

However, various means for performing image processing on image information captured by the image sensor S ₁ and detecting a straight line corresponding to the boundary L are configured using the control device 11 .

To explain automatic driving of the vehicle body V, as shown in FIG. The vehicle is steered based on the information detected by the image processing so that V automatically travels along the boundary _L1 between the uncut area B and the cut area C along the length direction of the work process. to be controlled, and the boundary L ₂ on the end side of one working stroke
In other words, as one reaches the opposite side of the uncut land B,
By repeating the automatic turn toward the starting end of the next working stroke that intersects with the previous working stroke, the lawn mowing work in a predetermined range is automatically performed in a so-called circular mowing style.

To explain the operation of the control device 11, when traveling is started, the rotation speed sensor
Every time you travel a set distance based on the detection information of S ₂ ,
Boundary detection processing, which will be described later, is performed, and based on the detected information, steering control is performed so that the vehicle body V automatically travels along the boundary _L1 .

Based on the detection information of the rotation speed sensor _S2 , as the distance corresponding to the length of one working stroke is within the set range, the boundary _L2 on the end side of the working stroke is determined. Perform the process to detect,
Based on the position information on the image, the end position of the vehicle body V relative to the current position is calculated, and the front and rear wheels 1 and 2 are maintained in the steering neutral state until the calculated position, thereby driving the vehicle straight ahead. After that, the robot turns for the next work process based on a preset and stored turn pattern.

Also, for example, by determining whether a preset number of strokes has been traveled or whether a preset distance has been reached, it is possible to determine whether the work is complete or not. In this case, the processes after the boundary detection process described above will be repeated. On the other hand, if the work is finished, the vehicle body V will be stopped and the entire process will be completed.

To explain the calculation of the deviation with respect to the boundary L ₁ on the stroke side, there is a deviation in the position in the width direction of the vehicle body and a deviation in the inclination with respect to the direction of the boundary L ₁ , but these are calculated as quantitative values. Alternatively, only the direction of deviation may be calculated.

Also, if we add an explanation about the steering control process,
The parallel steering type corrects positional deviations in the width direction of the vehicle body, and the four-wheel steering type corrects directional deviations. However, the steering amounts of the front and rear wheels 1 and 2 may be made different to correct the positional and orientation deviations at the same time.

Next, the boundary detection process will be described in detail based on the flowchart shown in FIG.

However, the boundary detection process described below is performed by utilizing the phenomenon that the unmoved area B appears darker than the mowed area C.

That is, as the boundary detection process is started, the image sensor S ₁ performs imaging processing, and the captured image information is adjusted to a preset pixel density (set to 32 x 32 pixels). Correspondingly quantized.

Next, 3
Using a mask for ×3 pixels, the process of calculating the differential value SX of the brightness change in the x-axis direction on the image is performed for each pixel arranged in the two-dimensional direction based on the following equation (i) ( (See Figure 3).

SX(x,y)=(c+2f+i)−(a+2d+g)…(i
) However, in this differentiation process, the sign of the differential value SX is determined based on whether the boundary L is located on the left or right side with respect to the vehicle body V, and a value that has either a positive or negative sign is determined. It is designed to only seek the following.

In other words, in equation (i) above, the brightness of the pixel located on the left side of the image is subtracted from the brightness of the pixel located on the right side of the image, so if the right side is brighter (see Figure 4), , the differential value SX
is a positive value, and if the left side is brighter (5th
(see figure) has a negative value.

Therefore, when driving with the boundary _L1 located on the right side of the vehicle body V, the differential value
When extracting a vehicle whose SX is a positive value and causing the vehicle to run while positioned on the left side, a vehicle whose differential value SX is a negative value will be extracted.

In addition, the differential value of the brightness change in this x-axis direction
The process of calculating SX corresponds to the differentiating means 100.

Then, by performing binarization so as to extract only pixels for which the absolute value of the differential value SX is greater than or equal to a preset threshold value, pixels at locations where brightness changes are large are extracted.

Note that this binarization process corresponds to the binarization means 101.

Next, the pixels extracted by the binarization means 101 are searched in a direction intersecting the boundary L and from dark to bright, and the pixels extracted by the binarization means 101 are
By extracting the first existing pixel among the pixels extracted in , the bright part, that is, the shadow A reflected in the above-mentioned mowed area C (see Figures 4 and 5)
Performs shadow removal processing to remove.

Note that the process of removing this shadow is carried out by the pixel extraction means 1.
This corresponds to 02.

Then, using Hough transform processing, for each pixel from which the shadow has been removed, a number of straight lines that pass through that pixel and are divided at each set angle are plotted in the polar coordinate system (ρ, θ) based on the following formula (ii). Project it onto the screen, make a histogram of the frequency of the same coordinate values, find a straight line with the maximum frequency from the histogram, and use that straight line as a boundary in the direction along the work process.
Detect as a straight line corresponding to L ₁ .

ρ=x·cosθ+y·sinθ (ii) Note that the process of finding a straight line using this Hough transform process corresponds to the straight line calculation means 103.

However, in this boundary detection process, the screen used for explanation is a virtual one, and the straight lines corresponding to each of the boundaries L ₁ and L ₂ are not actually drawn, but the lines that correspond to the maximum frequency The value ρ obtained by the above equation (ii) and the value of the angle θ that corresponds to the value ρ are used as information corresponding to the detected straight line. Then, in the process of calculating the deviation with respect to the boundary and calculating the end position, from those values ρ and θ,
By mapping to the coordinate system of the actual ground, the actual positions of the boundaries L ₁ and L ₂ with respect to the vehicle body V are calculated.

Then, by controlling the steering as described above based on the information mapped on the actual ground, the vehicle body V automatically travels along the boundary _L1 , and as it reaches the terminal side boundary _L2 . , the robot will be guided to automatically turn toward the starting end of the next working stroke.

By the way, when detecting the boundary _L2 on the terminal side, the brightness changes in the front and back direction with respect to the vehicle body V, and also changes from bright to dark from the top to the bottom on the screen. In addition, in the differential processing described above, if the differential value of the brightness change in the y-axis direction is found and used in place of the differential value SX in the x-axis direction, the terminal side boundary L ₂ can be detected with the same processing. can do.

To explain the determination of whether or not the terminal side boundary L ₂ has been detected, since the aforementioned boundary detection process is performed for each set distance, the terminal side boundary L ₂ detected in the current boundary detection process The position of
The detected straight line is determined by determining whether or not it is in the vicinity of the position where the set distance is added in the longitudinal direction of the vehicle to the position of the boundary _L2 on the terminal side detected in the previous boundary detection process. , it can be determined whether or not the boundary _L2 is on the terminal side.

By the way, since the image sensor S ₁ is installed so as to image the work area on the front side of the vehicle from diagonally above, the position changes on the screen of each boundary L ₁ and L ₂ obtained from the captured image information are as follows. , in inverse proportion to the distance to the actual positions of boundaries L ₁ and L ₂ with respect to the vehicle body V, and becomes smaller as the distance increases.

Therefore, in each of the determination of the terminal side boundary _L2 , the calculation of the distance to the terminal side boundary _L2 , and the calculation of the deviation from the detected stroke side boundary _L1 , the detected position on the screen is Corresponding to this, the actual position with respect to the vehicle body V is converted.

To explain the shadow removal process, as shown in _FIG .
Based on whether it is on the left or right side of the screen, the search direction for the brightness change is in the x-axis direction, which is the direction that intersects the boundary _L1 , and from dark to bright. It is determined whether the search is to be performed from the direction of (see FIGS. 4 and 5).

As shown in Fig. 4, the search process when the left side of the screen is uncut land B and is dark is explained below.In order to search from the left side of the screen to the right side, each line in the x-axis direction is The initial value of the coordinate value is set to 1, and scanning is performed pixel by pixel from dark to bright, that is, from left to right, until the coordinate value reaches 32, and the binarized pixels extracted by the binarizing means 101 are It is determined whether the value IP is "1" indicating that a binarized pixel exists.

As the first existing pixel for which the value IP of the binarized pixel becomes "1" is extracted, the value DP of the binarized pixel used as image information from which the shadow A has been removed is set to "1", The search in the x-axis direction for one line is completed.

When the search for one line is completed, the process of determining the value IP of the binarized pixel in the next line is repeated by adding one at a time until the coordinate value in the y-axis direction reaches 32.

On the other hand, as shown in Figure 5, if there is uncut land B that looks dark on the right side of the screen, the initial value of the coordinate value in the x-axis direction is set to the final coordinate value of 32, and the value By subtracting one from , the search direction is reversed left and right so that the search in the x-axis direction is performed in the opposite direction from right to left.

Therefore, the search process for one line is completed when the first existing pixel whose value IP of the binarized pixel is "1" is extracted. Even if a shadow A appears in the mown area C, the pixels corresponding to the shadow A are not extracted, and the shadow A is automatically removed.

Also, by performing this shadow removal process, the above 2.
Pixels after the first pixel whose value IP is "1" are also removed from the processing target, and the number of pixels to be processed by the linear calculation means 103 can be reduced. As a result, high-speed processing becomes possible.

[Another example]

In the above embodiment, the detection information of the rotation speed sensor S ₂ is used to exemplify the case where the operation is performed when the vehicle body V approaches the end of the working stroke.
For example, each time boundary detection processing is performed, end detection may be performed alternately or simultaneously.

Further, in the above embodiment, the binarization means 101 is
Although we have illustrated a case where binarization is performed based on the magnitude of the differential value SX in the axial direction, for example, the differential value in the y-axis direction may also be used to consider the direction of brightness change. Good too.

Furthermore, in the above embodiment, the case where Hough transform processing is used as the straight line calculating means 103 for calculating the straight line corresponding to the boundary L is illustrated, but the straight line calculating means 103
The specific configuration of can be changed in various ways.

Further, in the above embodiment, the present invention is used as a means for automatically driving a lawn mowing work vehicle, but the present invention can also be applied as a means for detecting the boundaries of various work areas. The specific configuration of each part can be changed in various ways.

Incidentally, although reference numerals are written in the claims section for convenient comparison with the drawings, the present invention is not limited to the structure shown in the accompanying drawings by the reference numerals.

[Brief explanation of drawings]

The drawings show an embodiment of the imaging type boundary detection device according to the present invention, and FIG. 1 is a flowchart of boundary detection processing, FIG. 2 is a flowchart of shadow removal processing, and FIG. 3 is an explanatory diagram of differential processing. 4 and 5 are explanatory diagrams of captured images, FIG. 6 is a block diagram showing the control configuration, FIG. 7 is an overall side view of the working vehicle, and FIG. 8 is an explanatory diagram of the working area. B... Untreated working area, C... Treated working area, L
... Boundary, S ₁ ... Imaging means, SX ... Differential value, 1
00...differentiation means, 101...binarization means, 10
2... Pixel extraction means, 103... Linear calculation means.

Claims (1)

[Claims] 1 Boundary L between untreated working area B and treated working area C
This is an imaging type boundary detection device that detects a straight line corresponding to the boundary L, and includes an imaging means _S1 that images the work area in a secondary direction, and a straight line that intersects the boundary L from the image taken by the imaging means _S1 . a differentiating means 100 for obtaining a differential value SX of each quantized pixel arranged in a two-dimensional direction based on a brightness change in the direction; and a differential value SX that is obtained.
A binarizing means 101 extracts pixels with a large value, and a binarizing means 101 searches in a direction intersecting the boundary L and from dark to bright, and extracts pixels from among the pixels extracted by the binarizing means 101. Pixel extraction means 102 for extracting the first existing pixel, and the pixel extraction means 10
An imaging type boundary detection device is provided with a straight line calculation means 103 for determining a straight line passing through the pixels extracted in step 2.