JPS63103382A - Erase point detector - Google Patents

Abstract

PURPOSE:To quickly and surely detect an erase point by setting a mask for each erase point candidate point obtained from a line cumulative image and using this mask to detect the erase point. CONSTITUTION:Edge information is extracted from the image, which is picked up by an image pickup part 201, by an edge calculating means 100 to calculate the direction of the edge. Lines are generated on the basis of this calculated edge direction by a line generating and accumulating means 101 and are accumulated as a line cumulative image. Erase point candidates are calculated in accordance with the accumulated line cumulative image by an erase point calculating part 213. A mask is set for each calculated erase point candidate point by an erase point detecting means 102, and the erase point candidate point of which the variance of the number of lines passing each side of the mask is smaller than a prescribed threshold and the total sum of lines passing respective sides of the mask is a maximum value is detected as the erase point.

Description

DETAILED DESCRIPTION OF THE INVENTION (Industrial use!?) The present invention relates to a vanishing point detection device that is attached to an automatic guided vehicle and used to identify three-dimensional objects and the like.

(Prior Art) In order to control the travel of an unmanned guided vehicle, etc., it is necessary to identify, for example, a three-dimensional object from a two-dimensional image captured by an imaging unit such as a television camera attached to an unmanned guided vehicle. Vanishing point detection technology is conventionally known for detecting and identifying the orientation and position of a three-dimensional object from a two-dimensional image.

Generally, when an object is captured as a two-dimensional image using a perspective method, rectangular parallelepiped objects 401, 40 as shown in FIG.
The apparent distance of two parallel ridge lines (hereinafter referred to as parallel lines) 403, 404 or 405, 406 decreases as the parallel lines move away from the camera, and finally reaches "0". This point DA at 0'' is called a "vanishing point." In other words, vanishing point detection technology detects the vanishing point from the apparent distortion that occurs as a result of imaging a three-dimensional object with depth as a two-dimensional image, and thereby detects information about the three-dimensional object, such as
This is a technology that detects the inclination and direction of the surface of an object.

To detect this vanishing point, KENDER first expressed the parameters of the linear equation in the two-dimensional image on a pseudo-Hough transform plane using HOUGH transform. Since one straight line in the real image space is represented as one point on the pseudo-Hough plane, the vanishing point in the real image space can be determined by searching for a straight line on the pseudo-Hough plane. KEND
In this way, ER further Hough-transformed the pseudo-Hough plane, defined a straight line on the pseudo-Hough plane, and detected the vanishing point DA. 0HTA et al.: As shown in Figure 57 (b), the distance ratio is derived from the area ratio when affine transformation is applied to the two texture components 407 and 408 in the screen, and this is used to determine the position of the vanishing point DB. For example, if you want to define the
27 pages, Katsushi Ikeuchi).

Once the vanishing point is determined in this way, the inclination of the plane of the three-dimensional object is determined from the vanishing point, and thereby information regarding the three-dimensional object can be obtained from the two-dimensional image.

However, in such conventional vanishing point detection methods, KENDER's algorithm requires two Hough transforms to detect the vanishing point, and the algorithm of 0HTA et al. Therefore, if there are many edge constituent points, a huge amount of calculation is required, and it takes a considerable amount of time to detect the vanishing point. There is a question of what we can or cannot do.

(Purpose and Structure of the Invention) The present invention has been made in consideration of the above points, and its purpose is to quickly detect a vanishing point. To achieve this purpose, the overall structure is shown in FIG. As shown, the imaging unit 201
The edge calculation means 100 extracts edge information from the image taken by the edge calculation means 100 to calculate the edge direction, and based on the calculated edge direction, the straight line generation and accumulation means 101 generates straight lines and accumulates them as a straight line cumulative image. , a vanishing point candidate point is calculated by the vanishing point calculation unit 213 from the accumulated straight line cumulative image, a mask is set for each vanishing point candidate point calculated by the vanishing point detection unit 102, and a passing straight line that passes through each side of the mask is set! ! The configuration is such that a vanishing point candidate point where the variation in the number of IT is smaller than a predetermined darkness value and where the total number of passing straight lines on each side of the mask has the maximum value is detected as a vanishing point.

(Example) The present invention will be explained below based on the drawings.

FIG. 2 is a block diagram of a vanishing point detection device according to the present invention.

First, to explain the configuration, this device includes an imaging unit 201 such as a television camera that converts light reflected from a three-dimensional object into a two-dimensional image signal, and an image memory that stores the two-dimensional image signal captured by the imaging unit 201. 202, a vertical edge extraction unit 203 that extracts the vertical component of an edge (hereinafter referred to as a vertical edge) from the two-dimensional image signal stored in the image memory 202, and a horizontal edge component of the edge (hereinafter referred to as a horizontal edge). ), an m-line processing unit 205 that thins the edges extracted by the vertical edge extraction unit 203 and the horizontal edge extraction unit 204;
An isolated area removal unit 206 removes isolated areas such as minute noise and short line segments from the screen after thinning processing, and an image memory 207 that stores the results of thinning and isolated area removal performed on the extracted vertical edges. , an image memory 208 that stores the results of thinning and isolated area removal for the extracted horizontal edges, and an edge direction that calculates the edge direction for all points constituting the edge from the results of the image memories 207 and 208. A calculation unit 209, a straight line generation unit 210 that generates a straight line with the slope of the edge calculated by the edge direction calculation unit 209, a straight line accumulation unit 211 that accumulates the straight lines generated by the straight line generation unit 210, An image memory 212 for storing images, and an image memory 21 for storing images.
Vanishing point candidate point unit 213 that calculates vanishing point candidate points from the contents of 2.
and an image memory 214 in which the calculated vanishing point candidate points are stored.
The vanishing point detection mask setting unit 215 sets an nxn vanishing point detection mask centered on the vanishing point candidate point of A passing straight line number calculating unit 216 that calculates a parameter group such as the number of straight lines and the total number thereof, and a passing straight line number calculating unit 216
a parameter storage section 217 that stores various parameter groups calculated by the parameter storage section 217, and a vanishing point calculation section 2 that detects a vanishing point using the parameter groups stored in the parameter storage section 217.
1B.

Here, the vanishing point candidate point calculation unit 213 calculates
D) The linear cumulative image G (x, y) in the image memory 212
(see the flowchart shown in FIG. 10 of the same-dated patent application ``Vanishing Point Detection Apparatus''), and calculates the maximum cumulative value A by multiplying this maximum cumulative value A by a predetermined coefficient δ. The linear cumulative image in the memory 212 is binarized and the binarized linear cumulative image G'(x, y) is stored in the image memory 214 as a vanishing point candidate point.

Further, various parameter groups stored in the parameter storage unit 217 include, as specifically shown in FIG. 5, the total number N of vanishing point candidate points, the The number of passing lines Up (X
+ 3'), L e (X + y), L, (x, y
) tR; (x*y), and the total number S (x, y) of the number of passing straight lines on each side of the mask.

The vanishing point calculation unit 218 determines the vanishing point, but since straight lines converge at the vanishing point from all directions, a mask set around the true vanishing point has a large number of lines on average on each side of the mask. A straight line is passing through it. Therefore, the vanishing point calculation unit 2
18 is a passing straight line aUP (x, y) on each side of the mask, Le
(x, y), Lo(x.

y), R+(x, y) is less than a certain threshold β, and the total number of passing straight lines on each side of the mask S(x, y)
The vanishing point candidate point that is the maximum is determined as the vanishing point.

Now, in the vanishing point detection device having the above configuration, the imaging unit 201
The two-dimensional image taken in from is temporarily stored in the image memory 202, and then sent to a vertical edge extraction section 203 and a horizontal edge extraction section 204, respectively.

A vertical edge extracting unit 203 and a horizontal edge extracting unit 204 calculate changes in the grayscale values in the horizontal and vertical directions of the two-dimensional image using predetermined vertical edge extraction differential operators and horizontal edge extraction differential operators (not shown), respectively.

The horizontal density value change obtained in the vertical edge extraction unit 203 indicates the strength of the vertical edge in the two-dimensional image, and the vertical density value change obtained in the horizontal edge extraction unit 204 indicates the strength of the vertical edge in the two-dimensional image. It shows the strength of the lateral edges.

The vertical edge extraction unit 203 and the horizontal edge extraction unit 204 further perform processing such as density conversion on the horizontal and vertical grayscale value change data of the two-dimensional image, binarize it, and use the binarized edge information as a retrace line. The data is sent to the conversion processing unit 205.

The thinning processing section 205 thins an edge having a certain width into an edge having a width "l" by well-known processing, and sends this to the isolated area removal section 206. An isolated area removal unit 206 removes isolated areas such as extremely short line segments and noise, and stores the thus preprocessed vertical edge and horizontal edge information in image memories 207 and 208, respectively.

Next, in the edge direction calculation section 209, the image memory 207.
The direction or slope of the edge is calculated from the vertical edge and horizontal edge information stored in the straight line generating unit 210.
A straight line is generated based on the slope of this edge, and the generated straight line is stored in the image memory 21 in the straight line accumulator 211.
It is accumulated and stored in 2. That is, the image memory 212 stores the cumulative value of the straight line.

The above processing from the imaging unit 201 to the image memory 212 is explained in detail in the above-mentioned patent application "Vanishing Point Detection Device" dated the same day, so it will not be explained in further detail here.

Now, the vanishing point candidate point calculation unit 213 calculates a vanishing point candidate point according to the procedure shown in FIGS.

That is, the vanishing point candidate point calculation unit 213 calculates the
A histogram of cumulative values as shown in FIG. 3(B) is created from the linear cumulative image G (x, y) accumulated in the image memory 212 as shown in FIG. The maximum cumulative value A is determined from this histogram and normalized so that it becomes "255". When the maximum cumulative value A is calculated, this maximum cumulative value A
is multiplied by a predetermined coefficient δ (0<δ<1) to provide a threshold value δ·A for binarizing the linear cumulative image G(x, y). Here, the coefficient δ is set between “0” and “0” depending on the operator of the device.
It can be set arbitrarily within the range of "l". FIG. 3(c) shows a binarization mask with the threshold value δ・A set in this way, and this mask is used to create a linear cumulative image G (x
, y) and the threshold value δ・A, G (x, y)>δ・
In the case of A, the binarized cumulative image G'(x, y) is set to "255
”, and when G(x, y)<δ・A, the binarized cumulative image G'(x, y) is set to "0".The linear cumulative image G' binarized in this way (x, y) is stored in the image memory 214. Of this binarized linear cumulative image G' (x, y), the one having the value "255" becomes the vanishing point candidate point.

Next, in the vanishing point detection mask 1 fixing unit 215, a point having a value of "255" of the binarized cumulative image G'(x, y) stored in the image memory 214, that is, a vanishing point candidate point is located as the center. A mask 301 of nXn is set. , FIG. 4 shows an example of a mask 301 set centering on the vanishing point candidate point PQ located at the coordinates (x, y) in this manner.

This mask 301 has a size n or 5". This mask 301 scans the image memory 214 from left to right and from top to bottom, and scans each vanishing point candidate point (x, y), the passing straight line number calculation unit 216 calculates various parameters.As described above, these parameters include the total number N of vanishing point candidate points, the X coordinate of each vanishing point candidate point, y coordinates, and the total number of passing lines U p (x, y), L that pass through the upper, left, lower, and right sides of the mask 301 set around each vanishing point candidate point, respectively.
e(x.

y)lo(x, y)+Rt(x, y), and the sum of passing straight lines on each side (x, y).

For example, the number U of straight lines passing through the upper side of the mask 301
p (x, y) is expressed by the general formula +H(x - [-] + 1, y - [-] +...
...+H(X+[] 1.y-[]). Here, H (x, y) is the number of straight lines passing through the point (X + y), [nl represents the maximum number that does not exceed n.

In the case of a mask where n is "5" as shown in FIG. '
) + Lo (x, y), Rr (x + y
) are respectively expressed by the following formulas. In other words, the number of passing straight lines passing through the upper side of the mask 301 Up (x, y)
is (x-1, y-2) + H (x, y-2) + H the number of straight lines passing through the left side of the mask 301.

(x, y) is (x-2, y-1) +H (x-2, 3/) +) (number of straight lines passing through the lower side of the mask 301 L0 (x, y
) is (x-1,y+2)+H(x,y+2)+H mask 30
The number of passing straight lines R1<x, y) passing through the right side of 1 is expressed as (x+2.y-1)+H(x+2.y)+H(x).

Also, the total number of passing straight lines on each side S (x, y) is S(x
+ y ) = Up(x + y ) ”Le(x
+ y) 10t, o(x, Y> +Ri(X, 'J)
It is required as.

The parameters calculated in this way are sequentially stored in the parameter storage section 217 as shown in FIG. In FIG. 5, the address of the parameter storage unit 217 is expressed in hexadecimal notation, and HEX#(n) is a symbol for converting n in decimal notation to hexadecimal notation. The contents of the parameter storage section 217 at address HEX well(n) are expressed as MR(n). In hexadecimal notation, the content MR(1) at address "l" stores the total number of vanishing point candidate points, and the content MR(2) at addresses "2" and '3'.

The X and Y coordinates of the vanishing point candidate point Q1 are stored in MR(3), respectively, at addresses "4" and "5".

6”, “7” contents MR(4), MR(5).

MR(6) and MR(7) respectively have numbers of passing straight lines Up(x, y), Le(x, y) on the upper, left, lower, and right sides of the mask set around the vanishing point candidate point Q1, Lo(
x, y), l(x, y) are stored, and the content MR(8) at address “8” contains the sum S (x, y
) is stored. Contents after address “9” MR (9)
, MR(A).

MR(B), -... have vanishing point candidate points Q2. ...X
Coordinates, y-coordinates, number of passing straight lines U on the top, left, bottom, and right sides
p(x, y), Le(x, y), Lo(x, y), R+
(x, y) and the total number of passing straight lines S (x, y) on each side are stored sequentially.

In this way, each vanishing point candidate point Ql is stored in the parameter storage unit 217.
, Q2. After storing the parameters related to ..., the vanishing point calculation unit 21B uses these parameters to determine whether the variation in the number of passing straight lines on each side of the mask at each vanishing point candidate point is less than or equal to a predetermined threshold value β. At the same time as the judgment, the sum S of passing straight lines on each side among the vanishing point candidate points that satisfy this condition is calculated.
A vanishing point candidate point where (x, y) is the maximum is detected, and this is calculated as the vanishing point. Note that the degree of variation in the number of passing straight lines on each side of the mask at each vanishing point candidate point is m a x
(U p (x + y), L e (x + y),
Lo(x + y) +R+(x, y)) -mi
n (Up(x, y), Le(x.

y), Lo(X, y), R+(x, y)). Here max (Up(x, y).

Le (x, y), Lo (x+ y), R; (x, y))
are UP (x, y), Le (x+ y), Lo (x, y)
, R, (x, y), and give m1n(UP(X, y), Le(x, y), Lo(X
,y).

R, (x, y)) is UP (x, y), Le (x, y).

Give the smallest number of passing straight lines among Lo(x, y) and J(x, y).

FIG. 6 shows in detail the flow of processing by the vanishing point calculation unit 218.

In FIG. 6, in step (F-1), the total number N of vanishing point candidate points is taken out from the parameter storage section 217. That is, the contents of MR(1) are extracted. Then step (F-2)
Then, when each vanishing point candidate point has a variation in the number of passing straight lines on each side of the mask or is less than a predetermined threshold β, an area P (1) for storing the total number S(x, y) of passing straight lines on each side of the mask. Or clear P (N). Note that regions P(1) to P(N) are prepared for the number N of vanishing point candidate points,
As explained below, "0" is stored in the area corresponding to the vanishing point candidate points whose dispersion is equal to or greater than the threshold β among p (i) to P(N), and the vanishing point candidate points whose dispersion is equal to or less than the threshold β In the area corresponding to the vanishing point candidate point, the total number S (x, y) of passing straight lines on each side of the mask is stored.

Now, in step (F-3), the parameter storage section 2
In step 17, a numerical value n' indicating which vanishing point candidate point is currently targeted is cleared. Note that the numerical value n' is expressed in lO base.

Next, in step (F-4), the numerical value n'' is increased by "1".In step (F-5), it is determined whether the numerical value n' has become larger than the total number N of vanishing point candidate points.

When the number n' is smaller than the total number N, the number n'
In order to investigate the variation in the number of passing straight lines on each side of the mask as described above for the vanishing point candidate points corresponding to
Proceed to F-6). On the other hand, in step (F-5),
When the number n' becomes larger than the total number N, the investigation of dispersion for all vanishing point candidates has been completed, so
Steps (F-9) to (F-1
Proceed to 0).

In steps (F-6) to (F-8), variations in the number of passing straight lines on each side of the mask are examined. First step (F-
In 6), the address (Add,) of the parameter storage unit 217 storing the number Up (x, y) of passing straight lines on the L side of the mask is calculated in hexadecimal notation for the n'-th vanishing point candidate point. This address is represented in decimal notation as (7n'-3) and in hexadecimal notation as HEX#(7n'-3). Next, in step (F-7), as described above, the variation in the number of straight lines passing through each side of the mask at the nth [1 vanishing point candidate point is max (MR(Add, ), MR(Add, +1),
MR (8aa, +z), MR (8dd.

+3)) -win (MR(Add, ), MR(
Add, +1), MR (8dd, +2).

MR(Add, +3) > is determined, and it is determined whether this variation is larger than a predetermined threshold value β. When the dispersion is larger than the threshold β, the n“th vanishing point candidate point cannot be a vanishing point, so
Since nothing is stored in the area P(n'), that is, P(n') is initially cleared in step (F-2), "0" is stored. Returning to step (F-4), the process proceeds to the next vanishing point candidate point. Step (F-7
) is smaller than the threshold β, then n
Since the ′th vanishing point candidate point can be a vanishing point, step (
Proceeding to step (F-8), the total number of straight lines passing through each side of the mask for this vanishing point candidate point S (x
, y) are stored in area P(n'). Then step (
Returning to F-4), the process proceeds to the first-order vanishing point candidate point.

In this way, steps (F-4) to (F-8) are repeated for each vanishing point candidate point, the contents of the region P(n') are determined, and the numerical value n' is determined in step (F-5). Step (
Proceed to F-9).

In the regions P (1) to P (N), the variation is the threshold β
, the total number of passing straight lines S (x, y) for that vanishing point candidate point is stored, and in step (F-9), the area P (1) to P (N) is The total number S (x.

Find the total number R that takes the maximum value among y), and calculate this total fiR
Find the number Z of the vanishing point candidate point with . In order to determine the Z-th vanishing point candidate point having the total number R as the vanishing point, in step (F-10), the X and Y coordinates of the second vanishing point candidate point are taken out from the parameter storage section 217. That is, X
The coordinate X is found from MR (Z-6), and the y coordinate Y is found from MR (
Z-5). This allows the vanishing point coordinates (x, y
) is calculated.

As described above, in this embodiment, the straight line cumulative image stored in the image memory 214 is binarized by the straight line cumulative image 211, and this is calculated as a vanishing point candidate point, and a predetermined size is set for each of these vanishing point candidates: 4n points. A mask 301 is set, parameters such as the number of passing directions Vj passing through each side of this mask are calculated, and then from these parameters it is determined that the variation in the number of passing straight lines passing through each side of the mask is less than a predetermined threshold value β. The total number of passing straight lines that are small and pass through each side of the mask S (x
.

Since the point where y) is maximum is detected as the vanishing point,
It is possible to detect a vanishing point in a short time, and at the same time, even if there are a large number of vanishing point candidate points, it is possible to detect a certain vanishing point with the highest probability among them.

(Effects of the Invention) As explained above, in the present invention, a mask is set for each vanishing point candidate point obtained from the straight line cumulative image, and the variation in the number of passing straight lines passing through each side of this mask is determined by a predetermined threshold value. Since the vanishing point candidate point that is smaller than , and has the maximum total number of passing straight lines on each side of the mask is detected as the vanishing point, the vanishing point can be detected quickly and reliably, making it possible to detect vanishing points quickly and reliably. This makes it possible to quickly control the vehicle's travel.

[Brief explanation of the drawing]

FIG. 1 is an overall configuration diagram of a vanishing point detection device according to the present invention, and FIG.
The figure is a block diagram of an embodiment of the vanishing point detection device of the present invention, FIG. 3 is a diagram showing the procedure for calculating a vanishing point candidate point in the vanishing point candidate point calculating section of FIG. 2, and FIG. 4 is a diagram showing a vanishing point candidate point calculation section. 5 is a diagram showing the contents of the parameter storage section in FIG. 2, FIG. 6 is a flowchart showing the process flow of the vanishing point calculation section in FIG. 2, and FIG. FIG. 7(a) is a diagram for explaining the concept of a vanishing point, and FIG. 7(b) is a diagram for explaining a conventional method of detecting a vanishing point. 201... Photography department, 202, 207, 208° 209
．． 212, 214--Image memory, 203--Vertical edge extraction unit, 204--Horizontal edge extraction unit, 205...
- Thinning processing unit, 206... Isolated area removal unit, 209
... Edge direction calculation section, 210 ... Straight line generation section, 2
DESCRIPTION OF SYMBOLS 11... Straight line accumulation part, 213-... Vanishing point candidate point calculation part, 215-... Mask setting part for vanishing point detection. 216... Passing straight line number calculation unit, 217... Parameter storage unit, 21B-... Vanishing point calculation unit Patent applicant: Nissan Motor Co., Ltd. Agent Patent attorney: Hiroshi Suzuki Figure 2 Figure 4... 301 Figure 5 Figure 7 ('r) (b)

Claims (1)

[Claims] an imaging unit that captures an image of an object; an edge calculation unit that calculates an edge direction based on edge information extracted from an image captured by the imaging unit; and a straight line based on the edge direction calculated by the edge calculation unit. a straight line generating accumulating means for accumulating the generated straight lines; a vanishing point candidate point calculating means for calculating a vanishing point candidate point from the straight line cumulative image accumulated by the straight line generating accumulating means; and a vanishing point candidate point calculating means for calculating the vanishing point candidate point. A mask is set for each vanishing point candidate point calculated by the method, and a mask is set for each vanishing point candidate point that has a variation in the number of straight lines passing through each side of the mask that is smaller than a predetermined threshold value and a total number of straight lines passing through each side of the mask has a maximum value. A vanishing point detection device comprising: a vanishing point detection means for detecting a point candidate point as a vanishing point.