JPH08166236A - Device for abstracting rectangle from twin-lens stereoscopic image - Google Patents

Abstract

PURPOSE: To abstract rectangles in a short time with small-scale circuit constitution by performing the detection of existence of edges and the detection of longitudinal edges and lateral edges in the edge detection range sought from the first rectangle information in case of seeking the second rectangle information. CONSTITUTION: An edge detector 27 performs the detection of existence of an edge in the second image within the range of having widened the rectangle shown by the first rectangle abstraction information by specified range, using the first rectangle abstraction information abstracted with a rectangle recognizer 5. A projector 28 performs projection within the range of having widened the rectangle shown by the first rectangle abstraction information by specified range, using the first rectangle abstraction information. In case of seeking the second rectangle abstraction information, the abstraction of rectangles can be performed in a shorter time than before by the amount of processing range being smaller than before because the second rectangle abstraction information is sought by performing the detection of the existence of edges and projection in the edge detection range sought from the first rectangle abstraction information. Furthermore, for the edge detector 27 and the projector 28, it becomes possible to materialize the scale of circuits where the range of detection of existence of edges and projection processing gets off small, in small size.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a rectangle extracting apparatus for twin-lens stereoscopic images. This rectangle extraction device is a twin-lens stereoscopic image captured by two image capturing devices (cameras) arranged vertically or horizontally when detecting the position of a front vehicle in an unmanned vehicle or the like and measuring a distance to a detected object. Using, the left and right (upper and lower) eyes extract a rectangle such as the back view of a car that is estimated to be the corresponding object (the same object), and the distance to the rectangle extracted using the parallax between both eyes in the triangulation procedure. A rectangle for measurement is extracted.

[0002]

2. Description of the Related Art FIG. 19 is a block diagram of a conventional rectangle extraction device for twin-lens stereoscopic images.

In FIG. 19, 1 and 2 are edge detectors,
3 and 4 are projectors, and 4 and 5 are rectangle recognizers. The first image and the second image, which are input to the edge detectors 1 and 2 and obtained by imaging with a camera (not shown), are left (upper) and right (lower) eye images, respectively, and are images at the same time.

In order to extract a rectangle, a method of detecting an edge is used here. In the edge detectors 1 and 2, masking is performed using, for example, a filter for detecting horizontal edges indicated by reference numeral 8 in FIG. 20 and a filter for detecting vertical edges indicated by reference numeral 9 in FIG. ,
The presence / absence of an edge for a certain pixel in the first and second images is detected with a threshold value as shown in the flowchart of FIG.

[0005] In step S1 of FIG.
Indicates the condition. In step S1, the presence / absence of edges is adjusted.
The pixel to be filled is a, and the threshold for checking the presence or absence of an edge is
TH ₁, Pixels around a are b to i. Also, a to i
The spatial arrangement of the
The filter E is shown by the matrix 12. Furthermore, the pixel
Let k be the filtering result of a.

As shown in step S3, the value calculated by the equation 13 shown in step S2 is the absolute value of k as the threshold value TH.
If it is judged to be larger than ₁ , that is, if YES, there is an edge shown in step S4, and if it is judged to be small, that is, NO, there is no edge shown in step S5.

After determining the presence or absence of an edge for each pixel in this way, projection is performed by the projectors 3 and 4 shown in FIG. The projection means, as shown in FIG. 22, in the first or second image G, for the edge 15 in the vertical direction (vertical direction), the number of edges for each line is counted and the histogram 16 is obtained.
Similarly, the histogram 17 is similarly created for the horizontal (horizontal) edge 17, and the mountain portions 19 and 20 of the curves of the histograms 16 and 18 and 21 and 22 form the vertical edge 15 and the horizontal edge 17, respectively. it indicates a portion having, in order to detect a vertical edge 15 and horizontal edge 17 provided with a threshold THv and TH _h.

The projection method will be described with reference to the flowcharts shown in FIGS. Step S1 of FIG.
Shows the filtering conditions. G for the first or second image
(Array of m pixels × n lines), horizontal edge projection histogram H _h (array of size n): all initial values 0, vertical edge projection histogram H _v (array of size n): all initial values 0, the threshold for the horizontal edge projection histogram is TH _h : all initial values are 0, and the threshold for the vertical edge projection histogram is TH _v : all initial values are 0.

In step S2, as shown in FIG. 22, i = 1 to m and j = 1 to n, and whether or not the pixel of the image G (i, j) of the i-th pixel j-th line has a vertical edge. to decide. This is because the image G shown in FIG. 22 is traced in the horizontal direction (1 to m direction) while the vertical direction (1 to m direction is
It is determined whether or not there is a vertical edge by performing an operation of lowering each pixel in the n) direction.

Only when the result of this judgment is YES, that is, when there is a vertical edge, the value of H _v (i) in the equation 24 is counted up in step S3. When the result of the determination is NO, that is, when there is no vertical edge, the process proceeds to step S4. Step S
4, it is determined whether or not the pixel of the image G (i, j) on the i-th pixel and the j-th line has a horizontal edge. This is because the image G shown in FIG. 22 has a horizontal edge by performing an operation of moving one pixel in the horizontal direction (1 to m direction) while tracing in the vertical direction (1 to n) direction. It is to judge whether or not.

Only when the result of this determination is YES, that is, when there is a horizontal edge, the value of H _h (j) in the equation 25 is counted up in step S5. If the determination result is NO, that is, if there is no horizontal edge, the process proceeds to step S6 in FIG. 24, and it is determined whether or not the value of Hv (i) is larger than the threshold value TH _{v for} the vertical edge. If this determination result is YES,
That is, if it is large, the vertical edge may be a part of the side of the rectangle. Therefore, in step S7, G (i, j) is a candidate pixel on the left and right sides of the rectangle.

If the decision result in the step S6 is NO, it is assumed in the step S8 that G (i, j) is not related to the candidates for the right and left sides of the rectangle. Next, in step S9, similarly, the value of H _h (j) for the lateral edge is
It is determined whether it is larger than the threshold value TH _h . If the result of this determination is YES, that is, if it is large, it may be part of the side of the rectangle, so in step S10 G
(I, j) is a candidate pixel on the upper and lower sides of the rectangle.

If the decision result in the step S9 is NO, in the step S11, G (i, j) is not related to the candidates for the upper and lower sides of the rectangle. After that, the rectangle recognizers 5 and 6 shown in FIG. 19 recognize the rectangles to check whether or not these rectangle candidates are really rectangles. The operation in this case will be described with reference to the flowchart of FIG.

The conditions for rectangle recognition are shown in step S1 of FIG. In step S1, the upper or lower side of the rectangle is H _j , the upper or lower side of the candidate is a, the left or right side of the rectangle is V _i , the left or right side of the candidate is b,
Further, the threshold is TH _k .

In steps S2 to S6, it is checked whether the end points of the respective candidate straight lines exist within a range of ± THk, and it is determined whether they are rectangular. That is, in step S2, ± T from the left end of H _j for j = 1 to a
It is determined whether or not the upper end of V _i (i = 1 to b) exists at the position of H _k . If the determination result is NO, step S
It is assumed that 3 is not rectangular.

If the result of the determination in step S2 is YES, in step S4, V _i ′ (i = 1 to 1 which is different from V _{i in} step S2 is located at ± TH _k from the right end of H _j of _j = 1 to a). Determine if the top of b) is present. If the determination result is NO, it is determined that the rectangle is not rectangular in step S3.

If the decision result in the step S4 is YES, in a step S5, H _j ′ (j = 1 different from H _{j in the} step S2 is set at a position ± TH _k from the lower end of V _i (i = 1 to b). It is determined whether or not the left end of (a) exists.
If the determination result is NO, it is determined that the rectangle is not rectangular in step S3.

If the decision result in the step S5 is YES, in a step S6, H _j ′ (j = 1 which is different from H _{j in the} step S2 is set at a position ± TH _k from the lower end of V _i (i = 1 to b). It is judged whether or not the right end of (a) exists.
If the determination result is NO, it is determined that the rectangle is not formed in step S3, and if the determination result is YES, the rectangle is formed in step S7.

When the rectangle recognizers 5 and 6 recognize that the rectangle is a rectangle, as shown in FIG.
And the second rectangle extraction information are output.

[0020]

By the way, in the above-mentioned conventional rectangle extracting apparatus for twin-lens stereoscopic video,
Rectangle extraction is performed on both eye images at the same time, and in the extraction step, information of each image (such as the position of the rectangle of each image) is not used.

As described above, in the conventional example, since the same processing is performed for each eye, a rectangle can be extracted only for one eye due to noise and the like, conditions such as contrast and brightness, and differences in camera characteristics. In this case, there is a problem that a rectangle suitable for measuring the distance of a vehicle ahead cannot be extracted.

Further, there is a problem that the circuit scale increases because the same processing is performed by both eyes. The present invention has been made in view of the above point, and in a twin-lens stereoscopic image capable of extracting a rectangle suitable for measuring the distance of a vehicle ahead with a small circuit configuration. An object is to provide a rectangle extraction device.

[0023]

FIG. 1 shows the principle of a rectangle extracting device for twin-lens stereoscopic images according to the present invention. The rectangle extracting device for the twin-lens stereoscopic image detects the presence or absence of an edge of the first and second images of the same object imaged by the twin-lens imaging means, and when there is an edge, the vertical and horizontal edges are detected by a projection process. When detecting and recognizing that the figure by the detected vertical and horizontal edges is a rectangle, the first and second rectangle information corresponding to the first and second images is obtained.

In the figure, 1 is a first edge detecting means for detecting the presence or absence of an edge in the first image. 2 is first
The projecting means detects the vertical and horizontal edges from the detection result of the presence of the edge by the first edge detecting means.

Reference numeral 5 is a first rectangle recognition means, which outputs the first rectangle information when it recognizes that the figure formed by the vertical and horizontal edges detected by the first projection means is a rectangle. Reference numeral 27 denotes a second edge detecting means for detecting the presence or absence of an edge of the second image in an edge detection range obtained by expanding the rectangle indicated by the first rectangle information output from the first rectangle recognizing means by a predetermined range.

Reference numeral 28 denotes a second projection means, which detects vertical and horizontal edges from the detection result of the presence of an edge by the second edge detection means 27 in the above-mentioned edge detection range. Reference numeral 6 denotes a second rectangle recognition means, which outputs the second rectangle information when it recognizes that the figure formed by the vertical and horizontal edges detected by the second projection means 28 is a rectangle.

[0027]

According to the present invention described above, when the second rectangle information is obtained, the presence / absence of edges and the detection of vertical and horizontal edges are performed in the edge detection range obtained from the first rectangle information to detect the second rectangle. Since the information is obtained, the process of detecting the presence or absence of an edge and the process of detecting the vertical and horizontal edges from the second image are reduced, and accordingly, the rectangle extraction can be performed in a shorter time than the conventional case.

Further, since the first rectangle information is used for edge detection and projection processing from the second image, when the second rectangle recognition means 6 obtains the second rectangle information, it is highly reliable, for example, forward. It is possible to extract a rectangle suitable for measuring the distance and the like of the car.

Further, in the second edge detecting means 27 and the projecting means 28, the range of edge presence / absence detection and vertical / horizontal edge detection processing is small, so that it is possible to realize a small circuit scale accordingly. .

[0030]

Embodiments of the present invention will be described below with reference to the drawings. FIG. 2 is a block diagram of a rectangle extracting apparatus for a twin-lens stereoscopic image according to a first embodiment of the present invention.
In this figure, parts corresponding to those of the conventional example shown in FIG. 19 are designated by the same reference numerals, and the description thereof will be omitted.

In FIG. 2, 27 is an edge detector, and 28 is an edge detector.
Is a projector. The edge detector 27 uses the first rectangle extraction information extracted by the rectangle recognizer 5 to detect the presence or absence of an edge of the second image within a range obtained by expanding the rectangle indicated by the first rectangle extraction information by a predetermined range. Is. The projector 28 uses the first rectangle extraction information and performs projection within a range obtained by expanding the rectangle indicated by the first rectangle extraction information by a predetermined range.

The operation of this rectangle extracting apparatus will be described with reference to the flowchart of FIG. As shown in step S1 of FIG. 3, the coordinates of the position of the rectangle extracted in the first image are indicated by reference numeral 29, and the upper left corner is a (k,
l), the upper right is b (m, l), the lower left is c (k, n), and the lower right is d (m, n). Based on these coordinates, in the second image G, ± β pixels in the horizontal direction and in the vertical direction, The range in which ± α lines are extended is defined as the edge detection range of the second image G.

That is, in step S2, i = k−β˜
The second image G when m + β and j = 1-α to n + α
The presence / absence of an edge is detected for (i, j). The method of edge detection in this edge detection range is the same as that described in the conventional example with reference to FIG.

Next, in step S3 of FIG. 3, i =
The edge detection result obtained by the edge detector 27 in step S2 is projected for k-β to m + β and j = 1 to α + n + α. That is, in the edge detection range described above, FIG.
Also, projection is performed in the same manner as described with reference to FIG.

Then, in step S4, step S
The rectangle extraction is performed by the rectangle recognizer 6 recognizing the rectangle in the same way as described with reference to FIG. 25 in the conventional example from the projection result obtained by the projector 28 in 3, ie, the vertical and horizontal edges. As a result, the second rectangle extraction information is output.

According to the first embodiment described above, when the second rectangle extraction information is obtained, the presence / absence of an edge is detected and projected in the edge detection range obtained from the first rectangle extraction information to extract the second rectangle. Since the information is obtained, the rectangle can be extracted in a shorter time than the conventional one because the processing range is small.

Further, since the first rectangle extraction information is used for the edge detection and the projection processing from the second image, when the second rectangle extraction information is obtained from the rectangle recognizer 6 of the second image processing system, it is reliable. The rectangle extraction information has high property. Therefore, it is possible to extract a rectangle suitable for measuring the distance or the like of the vehicle ahead.

Further, the edge detector 27 of the second image processing system
Further, in the projector 28, the range of detection of presence / absence of an edge and the projection process is small, so that it is possible to realize a small circuit scale accordingly.

Next, a second embodiment will be described with reference to FIG. However, in FIG. 4, parts corresponding to the respective parts of the first embodiment shown in FIG. 2 are designated by the same reference numerals, and description thereof will be omitted. The rectangle extracting apparatus of the second embodiment shown in FIG. 4 is adapted to adaptively change the threshold in the edge detector by the threshold determiner when the rectangle is extracted only by one eye.

In FIG. 4, reference numerals 31 and 32 are edge detectors, 33 is an evaluator, and 34 is a threshold value determiner. The edge detectors 31 and 32 are basically the same as those shown by reference numerals 1 and 27 in FIG. 2, but the edge detector 31 detects the presence / absence of an edge described in the conventional example with reference to FIG. In this case, the threshold value TH ₁ can be output, and the edge detector 32 can change the threshold value TH ₂ .

The threshold determiner 34 is for varying the threshold TH ₂ according to the number of edges forming each rectangle obtained from the first and second rectangle extraction information. Further, when only changing the threshold value TH ₂ by the threshold value determining unit 34, the evaluator 33 is not necessary, so it is omitted in the description of changing the threshold value and will be described later.

The variable threshold operation will be described with reference to the flowchart of FIG. The conditions for the variable threshold operation are shown in step S1 of FIG. In step S1, the threshold for edge detection of the first image is TH ₁ , and the threshold for edge detection TH of the second image is TH.
₂ (the initial value is TH ₁ = TH ₂ ), the number of edges (pixel number) detected in the first image (first rectangular extraction information) is e ₁ , and the number of edges detected in the second image (second rectangular extraction information) The number of edges (the number of pixels) is e _2, and the parameters for changing the threshold are u ₁ and u ₂ .

In the initial state, the threshold determiner 34 reads the thresholds TH ₁ and TH ₂ from the edge detectors 31 and 32, and sets the threshold TH ₂ of the edge detector 32 so that TH ₂ = TH _1. deep.

In step S2, the threshold value determiner 34 uses the number of edges e1 obtained from the first and second rectangle extraction information.
, E2 are compared. If the result of this comparison is YES, that is, if e ₁ is large, the number of edges in the second image is so small that there is a possibility that a rectangle cannot be extracted. Therefore, in step S 3, the threshold value determiner 35
Lowers the threshold value TH ₂ by u ₁ so that edge detection is performed.

On the other hand, if the comparison result is NO, that is, if e ₁ is small, there is a possibility that a rectangle cannot be extracted properly because there are too many edges.
As shown in 6, edge detection is performed by raising the threshold TH ₂ by u ₂ .

Next, the case where the evaluator 33 is used will be described. The evaluator 33 obtains an evaluation value (evaluation parameter) when the threshold determiner 34 determines the threshold from the first and second images.

The threshold variable operation using this evaluation parameter will be described with reference to the flowcharts of FIGS. 6 and 7.
Conditions for calculating the evaluation parameters are shown in step S1 of FIG. In step S1, the pixel value of the first image is G ₁ (m, n), the pixel value of the second image is G ₂ (m, n),
The average value of the pixel values of the first image is a ₁ , the average value of the pixel values of the second image is a ₂ , and the evaluation parameters are k ₁ and k ₂ .

The evaluator 33 calculates the average values a ₁ and a ₂ of the pixels for the first image and the second image, respectively, as shown in equations 37 and 38 in step S2, and then in step S3.
As shown in Expressions 39 and 40, the sum of absolute values of the values obtained by subtracting the pixel values from the average values a ₁ and a ₂ is the evaluation parameter k ₁ ,
Calculate as k ₂ .

In step S4, the conditions for the threshold value determiner 34 to perform the variable threshold value operation are shown. In step S4,
The edge detection threshold value of the first image is TH ₁ , the edge detection threshold value TH _{2 of} the second image, the number of edges (pixel number) detected in the first image (first rectangle extraction information) is e ₁ , the second The number of edges (the number of pixels) detected in the image (second rectangle extraction information) is e _2, and the parameters for changing the threshold are u ₁ , u ₂ , u ₃ , u ₄ , and are obtained in step S3. The evaluation parameters are k ₁ and k ₂ .

Next, in step S5, e ₁ > e ₂
If k ₁ > k ₂ , that is, if YES, the threshold value TH ₂
Is likely to be large, so the equation 4 shown in step S6
In 1, the edge detection is performed by lowering u ₁ .

If the decision result in the step S5 is NO, the process advances to a step S7. In step S7, e ₁ > e ₂
If k ₁ <k ₂ , that is, if YES, step S
Unlike in the case of 5, the parameters (contrast, etc.) of the camera are considered to be different from the conditions of step S5.
Edge detection is performed by lowering u _{2 in the} equation 42 shown in step S8.

If the decision result in the step S7 is NO, the process advances to a step S9. In step S9, e ₁ <e ₂
If k ₁ <k ₂ , that is, if YES, there is a possibility that the number of edges is too large to extract a rectangle. Therefore, in the equation 43 shown in step S10, the threshold TH ₂ is increased by u ₃ to detect edges. Try to do it.

If the decision result in the step S9 is NO, the process advances to a step S11. In step S11, e ₁ <
Similarly, when e ₂ and k ₁ > k ₂ , edge detection is performed by increasing u _{4 in the} equation 44 shown in step S12.

According to the second embodiment described above, the same effect as that of the first embodiment can be obtained, and the threshold value for determining the presence / absence of an edge can be changed to obtain rectangular extraction information with an appropriate number of detected edges. If it is larger or smaller than the reference value for, it can be corrected.

Further, when the threshold value is changed, the difference between the conditions of the cameras which have obtained the first and second images is obtained as an evaluation parameter, and the evaluation parameter is reflected in the threshold value change, which is better. The threshold value can be corrected and set.

Next, a third embodiment will be described with reference to FIG. However, in FIG. 8, parts corresponding to the respective parts of the second embodiment shown in FIG. 4 are designated by the same reference numerals, and description thereof will be omitted. The rectangle extracting apparatus of the third embodiment shown in FIG. 8 is based on the rectangle that can be extracted from the first image or the second image,
The threshold in the edge detector is adaptively changed by the threshold determiner.

In FIG. 8, 46 is an edge detector and 47 is an edge detector.
Is a projector, and 48 is a threshold value determiner. Edge detector 46
Has the same function as the edge detector 32, and the projector 47 has the same function as the projector 28. Further, the threshold determiner 48 is
When only one of the second rectangle extraction information and the second rectangle extraction information is obtained, the threshold of the edge detector that detects the edge of the image is varied with the obtained image as a reference.

The operation for determining the reference image when performing the threshold variable operation will be described with reference to the flowchart in FIG. As shown in step S1 of FIG. 9, the rectangle extraction result (first rectangle extraction information) in the first image G ₁ is g ₁ ,
The rectangle extraction result (second rectangle extraction information) in the image G ₂ is g ₂ .

In step S2, when the rectangle is extracted in g ₁ , the reference image is set to the first image G ₁ as shown in step S3, and in step S4, when the rectangle is extracted in g ₂ , the process proceeds to step S5. As shown, the reference image is the second image G ₂ .

According to the third embodiment described above, the same effect as that of the second embodiment can be obtained, and even if the rectangle is not extracted by one of the two, the threshold value of the extracted one is changed to extract an appropriate rectangle. It becomes possible to obtain information.

Next, a fourth embodiment will be described with reference to FIG. However, in FIG. 10, parts corresponding to the respective parts of the second embodiment shown in FIG. 4 are designated by the same reference numerals, and description thereof will be omitted.

The rectangle extracting apparatus of the fourth embodiment shown in FIG. 10 is such that the threshold in the projector is adaptively changed by the threshold determiner when the rectangle is extracted only by one eye.

In FIG. 10, 50 and 51 are projectors and 5
2 is a threshold value determiner. The projectors 50 and 51 are basically the same as those shown by reference numerals 3 and 28 in FIG.
The projector 50 can output the thresholds TH _h and TH _{v in the} case of detecting the vertical and horizontal edges by the projection described in the conventional example with reference to FIGS. 23 and 24, and the projector 51 can output the threshold values. TH _h and TH _v can be changed.

The threshold determiner 52 is for varying the thresholds TH _h and TH _v according to the number of vertical and horizontal edges forming each rectangle obtained from the first and second rectangle extraction information. Further, the threshold value T is determined by the threshold value determiner 52.
Since the evaluator 53 is not necessary when only changing H _h and TH _v , it is omitted in the description of changing the threshold and will be described later.

The variable threshold operation will be described with reference to the flowchart of FIG. The condition of the variable threshold operation is shown in step S1 of FIG. In step S1, the threshold value for projecting the first image is TH ₃ , the threshold value for projecting the second image is TH ₄ , and the projection (vertical direction) for the first image (first rectangle extraction information) is exceeded. And the number of horizontal edges) is f ₃ , the second
In the image (second rectangle extraction information), the number of projections (vertical and horizontal edges) exceeding the threshold is f _4, and the parameters for changing the threshold are u ₁₁ and u ₁₂ .

In step S2, the threshold value determiner 34 compares the numbers f ₃ and f ₄ of projections exceeding the threshold value obtained from the first and second rectangle extraction information with each other. If the result of this comparison is YES, that is, if f ₃ is large, it is possible that a rectangle cannot be extracted because the projection threshold TH ₄ in the second image is too large. The determiner 52 lowers the threshold value TH ₄ by u ₁₁ so that projection is performed.

On the other hand, if the comparison result is NO, that is, if f ₃ is small, the projection threshold TH ₄ may be too low to extract an appropriate threshold. Therefore, in step S 4, as shown in equation 55, The threshold value TH ₄ is increased by u ₁₂ so that projection is performed.

Next, the case where the evaluator 53 is used will be described. The evaluator 53 obtains an evaluation value (evaluation parameter) when the threshold determiner 52 determines the threshold from the first and second images.

The threshold variable operation using this evaluation parameter will be described with reference to the flowcharts of FIGS. 12 and 13. The conditions for calculating the evaluation parameters are shown in step S1 of FIG. In step S1, the pixel value of the first image is G ₁ (m, n), and the pixel value of the second image is G ₂ (m, n).
n), the average value of the pixel values of the first image is a ₁ , the average value of the pixel values of the second image is a ₂ , and the evaluation parameters are k ₁₁ and k ₁₂ .

The evaluator 53 calculates the average values a ₁ and a ₂ of the pixels for the first image and the second image, respectively, as shown in equations 56 and 57 in step S2, and then in step S3.
As shown in Expressions 58 and 59, the sum of absolute values of the values obtained by subtracting the pixel values from the average values a ₁ and a ₂ is evaluated parameter k ₁₁ ,
Calculate as k ₁₁ .

In step S4, conditions for the threshold value determiner 52 to perform the variable threshold value operation are shown. In step S4,
The threshold for projecting the first image is TH ₃ , the threshold for projecting the second image is TH ₄ , and the projection in the first image (first rectangular extraction information) exceeds the threshold (vertical and horizontal edges). Is f ₃ , the number of projections (vertical and horizontal edges) in the second image (second rectangle extraction information) exceeding the threshold is f _4, and parameters for changing the threshold are u ₁₁ , u ₁₂ , u ₁₃ ,
u ₁₄ , the evaluation parameters obtained in step S3 are k ₁₁ , k
₁₂

Next, in step S5, f ₃ > f ₄
When k ₁₁ > k ₁₂ , that is, when YES, the threshold value TH ₄
Is considered to be large, so the equation 6 shown in step S6
At 0, u _{11 is} lowered and projection is performed.

If the decision result in the step S5 is NO, the process advances to a step S7. In step S7, f ₃ > f ₄
If k ₁₁ <k ₁₂ , that is, if YES, step S
Unlike in the case of 5, the parameters (contrast, etc.) of the camera are considered to be different from the conditions of step S5.
The projection is performed by lowering u _{12 in the} equation 61 shown in step S8.

If the decision result in the step S7 is NO, the process advances to a step S9. In step S9, f ₃ <f ₄
If k ₁₁ <k ₁₂ , that is, if YES, there is a possibility that the number of projections exceeding the threshold value is too large and the rectangle is not extracted. Therefore, the threshold value TH in the expression 62 shown in step S10 is set.
Raise ₄ by u ₁₃ to perform projection.

If the decision result in the step S9 is NO, the process advances to a step S11. In step S11, f ₃ <
Similarly, in the case of f ₄ and k ₁₁ > k ₁₂ , projection is performed by increasing u _{14 in the} equation 63 shown in step S12.

According to the fourth embodiment described above, the same effect as in the first embodiment can be obtained, the threshold value for projection is changed, and the number of detected vertical and horizontal edges is set to the appropriate rectangular extraction information. If it is larger or smaller than the reference value for obtaining it, it can be corrected.

Further, when the threshold value is changed, the difference between the conditions of the cameras that have obtained the first and second images is obtained as the evaluation parameter, and the evaluation parameter is reflected in the threshold value change, which is better. The threshold value can be corrected and set.

Next, a fifth embodiment will be described with reference to FIG. However, in FIG. 14, parts corresponding to the respective parts of the fourth embodiment shown in FIG. 10 are designated by the same reference numerals, and description thereof will be omitted.

In the rectangle extracting apparatus of the fifth embodiment shown in FIG. 14, the threshold in the projector is adaptively changed by the threshold determiner on the basis of which rectangle can be extracted in the first image or the second image. It is the one.

In FIG. 14, reference numeral 65 is a projector and 66 is a threshold value determiner. The projector 65 has the same function as the projector 51. Further, when only one of the first and second rectangle extraction information is obtained, the threshold determiner 66 sets the threshold of the projector that projects the edge detection information of the image with the obtained image as a reference. It is variable.

The operation for determining the reference image when performing the variable threshold operation is as follows. First, the rectangle extraction result (first rectangle extraction information) in the first image G ₁ is g ₁ , and the rectangle extraction in the second image G ₂ is Let the result (second rectangle extraction information) be g ₂ .

Next, g₁When the rectangle is extracted in
If the reference image is the first image G₁And g ₂Where the rectangle is
When extracted, the reference image is the second image G₂And that's all
According to the described fifth embodiment, the same effect as the fourth embodiment can be obtained.
In addition to being obtained, even if one of the rectangles is not extracted, it is extracted
Appropriate rectangle extraction information by changing the projection threshold for the one that has been issued
Can be obtained.

Next, a sixth embodiment will be described with reference to FIG. FIG. 15 shows the configuration of the rectangle evaluation circuit. The rectangle obtained by the first and second rectangle extraction information output from the rectangle recognizers 5 and 6 in the above-described first to fifth embodiments is time t. Later, it is determined whether the rectangles are the same.

In FIG. 15, reference numerals 68 and 69 denote rectangular position detectors, which detect the rectangular position from the first and second rectangular extraction information and output it. 70, 71, 72, 7
A memory unit 3 delays output information of the rectangular position detectors 68 and 69. An evaluator 74 determines whether or not the rectangles are the same after time t.

Next, the operation of the rectangle evaluation circuit for evaluating whether or not the corresponding rectangle after the time t is the same object as the rectangle before the time t will be described with reference to the flowcharts of FIGS. 16 and 17.

As shown in step S1 of FIG. 15, the first
Image or the second image G, the first image or the second image G _t after time t, the upper left of the recognized rectangular coordinates a (k,
l), the upper right and b (m, l), the lower left and c (k, n), and the lower right d (m, n) and the upper left of a _t (k of recognized rectangle coordinates after a time t, l ), The upper right corner is b _t (m, l), the lower left corner is c _t (k, n), and the lower right corner is d _t (m, n), the threshold of the rectangle ratio is TH _pq , the aspect ratio of the rectangle is p, and the rectangle Is q, the threshold value of the same rectangle determination is TH _r , and the calculation result is V.

In this rectangle evaluation circuit, the pixel value inside the rectangle in G (the portion surrounded by a to d) and the pixel value inside the rectangle in Gt (the portion enclosed in at to dt) corresponding to the pixel value If the sum of absolute values is less than or equal to the threshold value, they are regarded as the same rectangle.

According to the equations 76 and 77 shown in step S2, the vertical ratio p of the rectangle size after the time t and the lateral ratio q are obtained.
Find out. Next, in step S3, p <1 and q
> 1 or p> 1 and q <1 or | p−q |> TH _pq
If the values of p and q are not similar as a result of the judgment of step S4
In, it is determined that they are not the same rectangle.

If not, the average value of the aspect ratio p and the aspect ratio q is calculated as shown by the expression 78 in step S5 of FIG. 17, and whether the calculation result p'is greater than 1 in step S6. Therefore, as shown in steps S7 and S8, sub-sampling is performed according to the size of the smaller rectangle of G and Gt.

Next, equation 7 shown in steps S9 and S10
9 and 80, subsampled images G ′, G
The miscellaneous position sum V of the difference between t ′ and Gt, G is taken, and if V is larger than the threshold value TH _r in step S11,
In step S12, it is considered that they are not the same rectangle, and otherwise they are made the same rectangle in step S13.

Another method for determining the same rectangle will be described with reference to the flowchart of FIG. Step S of FIG.
1, the first image or the second image is G, and the time t
The subsequent first or second image is G _t , the upper left corner of the recognized rectangle is a (k, l), the upper right corner is b (m, l), the lower left corner is c (k, n), and the lower right corner is d (m, n), the upper left corner of the rectangle recognized after time t is a _t (k, l), the upper right corner is b _t (m, l), the lower left corner is c _t (k, n), the lower right corner. D
Let _t (m, n) be the threshold value of the same rectangle determination, TH _r , and the autocorrelation value be O, O _t .

[0092] As shown in Equation 81 and 82 in step S2, G, autocorrelation value O of the rectangular interior of the pixel values extracted in Gt, calculates the value of Ot, in step S3, the O-O _t Whether the absolute value is larger than TH _r , that is, whether O and Ot are the same is determined using the threshold value TH _r .

The rectangle evaluation circuit of the sixth embodiment can be used not only for determining the rectangle after the time t but also for determining the rectangle of the first image and the second image. According to the sixth embodiment described above, it is possible to determine whether the rectangles obtained in the first to fifth embodiments are the same after the time t.

[0094]

As described above, according to the present invention,
If the rectangle is extracted only in one eye, it can be extracted by changing the threshold adaptively, after extracting the rectangle in the one eye image, the other eye image in the vicinity of the position of the extracted rectangle The circuit scale can be reduced because it is performed in. Therefore, it is possible to extract a rectangle suitable for measuring the distance and the like of the vehicle ahead with a small-scale circuit configuration.

Further, although it has not been possible to automatically determine whether or not the rectangles after the time t are the same, it is possible to automatically determine.

[Brief description of drawings]

FIG. 1 is a principle diagram of the present invention.

FIG. 2 is a block diagram of a rectangle extracting device for twin-lens stereoscopic video according to a first embodiment of the present invention.

FIG. 3 is a flowchart for explaining the operation of FIG.

FIG. 4 is a block diagram of a rectangle extracting device for twin-lens stereoscopic video according to a second embodiment of the present invention.

5 is a flowchart for explaining the operation of FIG.

FIG. 6 is a first flowchart for explaining another operation of FIG.

FIG. 7 is a second flowchart for explaining another operation of FIG.

FIG. 8 is a block diagram of a rectangle extracting device for twin-lens stereoscopic video according to a third embodiment of the present invention.

9 is a flowchart for explaining the operation of FIG.

FIG. 10 is a block diagram of a rectangle extracting device for twin-lens stereoscopic video according to a fourth embodiment of the present invention.

11 is a flowchart for explaining the operation of FIG.

12 is a first flowchart for explaining another operation of FIG.

13 is a second flowchart for explaining another operation of FIG.

FIG. 14 is a block diagram of a rectangle extracting device for twin-lens stereoscopic video according to a fifth embodiment of the present invention.

FIG. 15 is a block configuration diagram of a rectangle evaluation circuit of a rectangle extracting device for a twin-lens stereoscopic image according to a sixth embodiment of the present invention.

16 is a first flowchart for explaining the operation of FIG.

17 is a second flowchart for explaining the operation of FIG.

18 is a flowchart illustrating another operation of FIG.

FIG. 19 is a block diagram of a rectangle evaluation circuit of a rectangle extraction device for a twin-lens stereoscopic image according to a conventional example.

FIG. 20 is a diagram showing an example of an edge detection filter.

FIG. 21 is an explanatory flowchart of edge presence / absence determination.

FIG. 22 is a projection explanatory diagram.

FIG. 23 is a first flowchart illustrating a projection method.

FIG. 24 is a second flowchart for explaining the projection method.

FIG. 25 is a flowchart for explaining rectangle determination.

[Explanation of symbols]

 1 1st edge detection means 3 1st projection means 5 1st rectangle recognition means 6 2nd rectangle recognition means 27 2nd edge detection means 28 2nd projection means

Claims (11)

[Claims] 1. The presence / absence of an edge of the first and second images of the same object imaged by the twin-lens imaging means is detected, and if there is an edge, the vertical and horizontal edges are detected by projection processing, and this is detected. A rectangle extracting device in a two-lens stereoscopic image, which obtains first and second rectangle information corresponding to the first and second images when recognizing that a figure by vertical and horizontal edges is a rectangle, the first image Edge detecting means for detecting the presence or absence of an edge, a first projecting means for detecting vertical and horizontal edges from the detection result of the edge existence of the first edge detecting means, and a vertical direction detected by the first projecting means. And a first rectangle recognition unit that outputs first rectangle information when it is recognized that the figure by the horizontal edge is a rectangle, and an edge detection range obtained by expanding a rectangle indicated by the first rectangle information by a predetermined range. With edge A second edge detecting means for detecting a vertical edge and a horizontal edge based on a detection result of the edge of the second edge detecting means in the edge detecting range, and a second projecting means for detecting the vertical and horizontal edges. And a second rectangle recognition means for outputting the second rectangle information when it is recognized that the figure by the vertical and horizontal edges is a rectangle.
A rectangle extraction device for eye-based stereoscopic images. 2. The edge detection range uses coordinates of four corners of a rectangle indicated by the first rectangle information, and the second coordinates are obtained from each coordinate.
± β pixels horizontally and ± α vertically within the image range
2. The rectangle extracting device for a twin-lens stereoscopic image according to claim 1, wherein the range is a line increased / decreased range. 3. The number of edges forming a rectangle is detected from the first and second rectangle information and compared, and the first edge number detected from the first rectangle information is detected from the second rectangle information. A first threshold value determining means is provided for controlling the threshold value used by the second edge detecting means to detect the presence or absence of the edge when the number is larger than the second edge number and increasing the threshold value when the number is smaller. The rectangle extraction device for a twin-lens stereoscopic image according to claim 1 or 2. 4. An evaluation means is provided for obtaining an average value of the number of pixels for each of the first and second images, and obtaining an sum of absolute values of values obtained by subtracting pixel values from the average value as an evaluation value. 4. The rectangle extracting device for twin-lens stereoscopic video according to claim 3, wherein the value is added as a parameter when the first threshold value determining unit performs the variable control of the threshold value. 5. When the first threshold value determining unit can obtain only one of the first and second rectangle information, an image corresponding to the obtained rectangle information is set as a reference image of the reference image. The rectangle extraction device for a twin-lens stereoscopic image according to claim 3 or 4, further comprising a function of varying a threshold value of an edge detection unit that performs edge detection. 6. From the first and second rectangle information, the first
And detecting and comparing the number of vertical and horizontal edges exceeding the projection threshold used when the second projection means detects the vertical and horizontal edges, and exceeding the projection threshold detected from the first rectangle information. When the number of the first vertical and horizontal edges is greater than the number of the second vertical and horizontal edges exceeding the projection threshold detected from the second rectangle information, the projection threshold of the projection unit is reduced, and when the number is small The rectangle extracting apparatus for a twin-lens stereoscopic image according to claim 1 or 2, further comprising: a second threshold value determining unit that performs control to increase a projection threshold value. 7. An evaluation means is provided for obtaining an average value of the number of pixels for each of the first and second images and for obtaining a sum of absolute values of values obtained by subtracting pixel values from the average value as an evaluation value. 7. The rectangle extracting apparatus for a twin-lens stereoscopic image according to claim 6, wherein the value is added as a parameter when the second threshold value determining unit variably controls the projection threshold value. 8. When the second threshold value determining means can obtain only one of the first and second rectangle information, an image corresponding to the obtained rectangle information is set as a reference image of the reference image. 8. The rectangle extracting device for a twin-lens stereoscopic image according to claim 6 or 7, further comprising a function of varying a projection threshold value of a projecting unit that detects the vertical and horizontal edges based on an edge detection result. 9. A rectangle position detector for detecting a rectangle of the first and second rectangle information at a predetermined time, a delay unit for delaying a rectangle output from the rectangle position detector, and the rectangle position detector. Evaluation for determining whether or not the rectangle output from the same is the same as the rectangle delayed by the delay means by comparing that the average value of the pixel values inside each rectangle is the closest. Means and means are provided.
8. A rectangle extracting device for twin-lens stereoscopic video according to any one of items 8. 10. The determination as to whether the rectangle and the previous rectangle are the same is performed by taking the norm of the pixel values inside the rectangle as the image value of the corresponding position inside the rectangle of the previous image and minimizing the norm. 10. The method according to claim 9, wherein the recognition is performed.
A rectangle extraction device for eye-based stereoscopic images. 11. The determination as to whether the rectangle and the previous rectangle are the same is performed by recognizing that the autocorrelation of the pixel values inside the rectangle takes the closest value. Rectangle extracting device for two-lens type stereoscopic image.