JP2003294422A - Object recognition apparatus and method therefor - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To accurately recognize an object including two straight lines arranged parallelly at a known interval. <P>SOLUTION: An object recognition apparatus has a camera 2 for outputting a picked-up image; a distance sensor for outputting two-dimensional distribution of a distance as distance data; a straight line extraction part 11 for extracting a pair of two-dimensional straight lines becoming a candidate of the object in a picked-up image plane; a calculation part for calculating a three-dimensional straight lines on the real space respectively corresponding to the two-dimensional straight lines, on the basis of the distance data to judge whether a pair of the three-dimensional straight lines are parallel and calculating the interval between a pair of the three-dimensional straight lines, when a pair of the three- dimensional straight lines are judged to be parallel; and an object judging part 13 for comparing the interval calculated by the calculation part with the judge value corresponding to the actual interval of the object to judge, whether a pair of the two-dimensional straight lines becoming the candidate of the object are the object. <P>COPYRIGHT: (C)2004,JPO

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an object recognizing apparatus and an object recognizing method for recognizing a pair of linear objects arranged in parallel at known intervals.

[0002]

2. Description of the Related Art In recent years, a monitoring device for monitoring a situation within a predetermined monitoring area has been noticed and put into practical use. This monitoring device is mounted on a moving body such as a vehicle or an aircraft, or is fixed to a stationary body such as a column. Regarding a railroad crossing monitoring device which is an example of the latter, conventionally, a monocular camera is used to image a railroad crossing including a rail of a railroad track (hereinafter referred to as “railroad rail”), and the position of the railroad rail on a captured image plane is specified. Then, based on the position of the railroad rail, it is monitored whether or not an obstacle such as a car or a person has entered the railroad crossing. The specification of the position of the railroad rail is important for specifying the railroad crossing area or for detecting the positional deviation of the camera.

[0003]

When only a monocular camera is used as a sensor for monitoring, an object other than a railroad rail, such as a road, a building, or a railway overhead wire, is erroneously recognized as a railroad rail. There is. The captured image obtained by the monocular camera is two-dimensional plane information, and the contour of the object including the railroad rail is approximately projected as a straight line. Generally, a straight line element on the image plane can be extracted by using a well-known straight line extraction method. However, from only the two-dimensional image plane,
It is not possible to accurately obtain three-dimensional information, that is, real space information including the distance (depth) to the object. Therefore, it is difficult to accurately evaluate the distance between two parallel straight lines (railroad rails). As a result, in a monitoring device using only a monocular camera, like a railroad rail, an object in which a pair of linear elements are arranged in parallel at a known interval,
It was difficult to recognize with high accuracy.

The present invention has been made in view of such circumstances, and an object thereof is to accurately recognize an object including two straight lines arranged in parallel at a known interval.

Another object of the present invention is to improve the reliability of the monitoring device by improving the recognition accuracy of the object.

[0006]

In order to solve such a problem, the first invention provides an object recognition apparatus for recognizing a pair of linear objects arranged in parallel at a known interval. This recognition device captures a scene including a monitoring area and outputs a captured image, a first camera, a distance sensor that outputs a two-dimensional distribution of distances in the monitoring area as distance data, and a captured image plane. , A straight line extraction unit that extracts a pair of two-dimensional straight lines that are candidates for the object, and based on the distance data, calculates a three-dimensional straight line in the real space corresponding to each of the two-dimensional straight lines, and a pair of three-dimensional straight lines Is determined to be parallel, and when it is determined that the pair of three-dimensional straight lines are parallel, a calculation unit that calculates the distance between the pair of three-dimensional straight lines, and the distance calculated by the calculation unit. An object determination unit that determines whether or not the pair of two-dimensional straight lines that are candidates for the object are objects by comparing with a determination value corresponding to the actual interval of the object.

The second invention provides an object recognition device for recognizing a pair of linear objects arranged in parallel at a known interval. This recognition device captures a scene including a monitoring area and outputs a captured image, a first camera, a distance sensor that outputs a two-dimensional distribution of distances in the monitoring area as distance data, and a captured image plane. A straight line extraction unit that extracts a pair of two-dimensional straight lines that are candidates for the object, and a vanishing point regarding the pair of two-dimensional straight lines in the captured image plane, and based on the vanishing points, intersects the pair of two-dimensional straight lines orthogonally. Calculating a vanishing point relating to the two-dimensional straight line as an orthogonal vanishing point, a two-dimensional straight line passing through the orthogonal vanishing point intersecting each of the pair of two-dimensional straight lines is calculated as a first intersection point and a second intersection point, A calculation unit that calculates a space in the real space between the first intersection and the second intersection based on the distance data;
An object for determining whether or not a pair of two-dimensional straight lines that are candidates for the object are objects by comparing the interval calculated by the calculator and a determination value corresponding to the actual interval of the object. It has a thing judging part.

Here, in the second invention, it is preferable that the first camera picks up an image of a pair of linear objects in an oblique direction.

Further, in the first or second aspect of the invention, a second camera for picking up a scene including the monitoring area and outputting the picked-up image is further provided, and the distance sensor includes the first camera and the second camera. It is preferable to include a stereo camera composed of

In the first or second invention, the object may be a rail of a railroad track.

A third aspect of the present invention is based on a captured image of a landscape including a surveillance area and distance data indicating a two-dimensional distribution of distances within the surveillance area. An object recognition method for recognizing a linear object is provided. This recognition method is a step of extracting a pair of two-dimensional straight lines that are candidates for an object in a captured image plane, and calculating three-dimensional straight lines in the real space corresponding to each of the two-dimensional straight lines based on the distance data. Step, a step of determining whether the pair of three-dimensional straight lines are parallel, and a step of calculating the interval between the pair of three-dimensional straight lines when it is determined that the pair of three-dimensional straight lines are parallel, By comparing the calculated interval and the determination value corresponding to the actual interval of the target object, it is possible to determine whether or not the pair of two-dimensional straight lines that are candidates for the target object is the target object. .

A fourth aspect of the present invention is based on a captured image of a landscape including a surveillance area and distance data showing a two-dimensional distribution of distances within the surveillance area. An object recognition method for recognizing a linear object is provided. This recognition method is based on the steps of extracting a pair of two-dimensional straight lines that are candidates for an object in the captured image plane, calculating a vanishing point for the pair of two-dimensional straight lines in the captured image plane, and based on the vanishing point. And calculating a vanishing point relating to a two-dimensional straight line orthogonal to the pair of two-dimensional straight lines as an orthogonal vanishing point, and a point at which the two-dimensional straight line passing through the orthogonal vanishing point intersects each of the pair of two-dimensional straight lines. Of the first intersection and the second intersection, based on the distance data, and the calculated distance and the target And a step of determining whether or not the pair of two-dimensional straight lines that are candidates for the object are objects by comparing with a determination value corresponding to the actual interval of the object.

[0013]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS (First Embodiment) FIG. 1 is a block diagram of an object recognition apparatus according to this embodiment.
The object recognition device 1 monitors a passing vehicle or a pedestrian who has entered the railroad crossing area projected in the monitoring area by using a stereo camera as a monitoring sensor. The stereo camera is fixed to a column or the like, and as shown in FIG. 2, images a railroad crossing including a railroad rail from diagonally above. In the captured image shown in the figure, in addition to the railway rails that are the objects to be detected, straight roads, horizon, and the like are also projected, and therefore, there are many straight line elements. The stereo camera has a pair of cameras 2 and 3 arranged at a predetermined interval (camera base line length), and each of the cameras 2 and 3 has an image sensor such as a CCD or CMOS sensor built therein. . The main camera 2 captures a reference image (right image) required when performing stereo image processing, and the sub camera 3 captures a comparative image (left image). The analog images output from the cameras 2 and 3 in a mutually synchronized state are digital images of a predetermined brightness gradation (for example, gray scale of 256 gradations) by the A / D converters 4 and 5. Is converted to. The image correction unit 6 corrects the brightness of the digitized image and performs geometric conversion of the image. Usually, the mounting positions of the pair of cameras 2 and 3 have an error to some extent, but there is an error, so that the left and right images are misaligned. In order to correct this shift, geometric transformation such as image rotation and parallel movement is performed using affine transformation or the like.

Through such image processing, reference image data is obtained from the main camera 2 and comparison image data is obtained from the sub camera 3. These image data are a set of brightness values (0 to 255) of each pixel included in the individually set effective image area. In the i-j coordinate system on the image plane, the lower left corner of the image is the origin, the horizontal direction is the i coordinate axis, and the vertical direction is the j coordinate axis. Both image data corresponding to one frame which is the minimum display unit of one image is stored in the image data memory 9 at any time.

The stereo image processing section 7 calculates distance data for the picked-up image corresponding to one frame based on the reference image data and the comparison image data. Here, the “distance data” is a set of parallax d calculated from a captured image corresponding to one frame, and each parallax d is associated with a position (i, j) on the image plane. In other words, the distance data is a two-dimensional distribution of distances within the surveillance area. Each parallax d is calculated once for each pixel block of a predetermined area (for example, 4 × 4 pixels) forming the reference image. FIG. 3 is an explanatory diagram of pixel blocks set in the reference image. The reference image area of pixels is divided into a matrix of pixel blocks PBij of 4 × 4 pixels. A parallax group corresponding to the number of pixel blocks PBij can be calculated from a captured image corresponding to one frame. As is well known, the parallax d is a horizontal shift amount with respect to the pixel block PBij that is the calculation target, and has a large correlation with the distance to the target object projected on the pixel block PBij. That is, the pixel block PB
The closer the object imaged in ij is to the cameras 2 and 3, the larger the parallax d of this pixel block PBij, and the farther the object is, the smaller the parallax d (infinitely far,
The parallax d becomes 0).

When calculating the parallax d for a certain pixel block PBij (correlation source), a region (correlation destination) having a correlation with the luminance characteristic of this pixel block PBij is specified in the comparison image. As described above, the distance from the cameras 2 and 3 to the object appears as a horizontal shift amount between the reference image and the comparison image. Therefore, when searching for the correlation destination in the comparative image, it is sufficient to search on the same horizontal line (epipolar line) as the j coordinate of the pixel block Pij that is the correlation source. The stereo image processing unit 7 sequentially shifts the correlation between the correlation source and the correlation destination candidate while shifting the pixel on the epipolar line pixel by pixel within a predetermined search range set based on the i coordinate of the correlation source. Evaluate (stereo matching). Then, as a general rule, the horizontal shift amount of the correlation destination (one of the correlation destination candidates) determined to have the highest correlation is set as the parallax d of the pixel block PBij.

The correlation between two pixel blocks can be evaluated by calculating the city block distance CB, for example. Formula 1 shows the basic form of the city block distance CB. In the equation, p1ij is the luminance value of the ij-th pixel of one pixel block, and p2ij is the ij-th luminance value of the other pixel block. The city block distance CB is the total sum of the difference (absolute value) between the pair of brightness values p1ij and p2ij corresponding to each other in the entire pixel block. The smaller the difference, the greater the correlation between the two pixel blocks. There is.

CB = Σ | p1ij-p2ij |

Basically, of the city block distances CB calculated for each pixel block existing on the epipolar line, the pixel block having the smallest value is determined to be the correlation target. The amount of deviation between the correlation destination and the correlation source thus identified is the parallax d. Note that the hardware configuration of the stereo image processing unit 7 for calculating the city block distance CB is disclosed in JP-A-5-114099, so refer to it if necessary. The distance data (i, j, d) calculated through such processing is stored in the distance data memory 8 at any time.

The microcomputer 10 includes a CPU and R
Although it is composed of an OM, a RAM, an input / output interface, etc., when it is functionally grasped, the straight line extraction unit 11,
It has an interval calculation unit 12, an object determination unit 13, and a monitoring unit 14. The straight line extraction unit 11 reads the reference image data from the image data memory 9 and extracts the straight line elements existing on the reference image plane. Then, the straight line extraction unit 11 identifies arbitrary two straight lines L1 and L2 (correctly, parameters defining the two straight lines L1 and L2 on the reference image plane) that are candidates for the railroad rail, and the two straight lines L1. , L2 are output to the interval calculation unit 12. The distance calculation unit 12 reads out the distance data from the distance data memory 8 and simultaneously
The distance D between 1 and L2 is calculated, and this distance D is output to the object determination unit 13. The object determination unit 13 compares the distance D with a predetermined set value D0 to determine whether or not the two straight lines L1 and L2, which are railroad rail candidates, are railroad rails. When the object determining unit 13 determines that the rail is a rail, the target determining unit 13 determines the position of the rail by the monitoring unit 14.
When it is determined that the railroad rail is not a railroad rail, the straight line extraction unit 11 is instructed to identify a new railroad rail candidate. The monitoring unit 14 sets a railroad crossing area from the position of the specified railroad rail, and monitors a pedestrian or a pedestrian who has entered this area.

FIG. 4 is a flow chart showing the procedure for recognizing the rail. First, in step 1, the straight line extraction unit 11 reads the reference image data corresponding to one frame, and the straight line elements existing on the reference image plane, that is,
Extract all (or any) linear elements that make up the rails, roads and horizon. Then, as shown in FIG. 5, the straight line extraction unit 11 specifies two straight lines L1 and L2 (two-dimensional straight lines) that are candidates for railroad rails in any combination from the extracted straight line elements.

A straight line element is extracted by a known straight line extraction method,
For example, it is possible by using methods such as image differential / binarization processing, Hough conversion, template matching, and the like. The Hough transform as an example is based on the logic that "a straight line can be uniquely determined by using the distance (ρ) from the origin and the inclination (θ) viewed from the X axis as parameters." , Ρ, and θ as axes, and conversion into a Hough space (parameter space). This Hough transform is
All the coordinate points on the Hough space showing all the straight lines passing through the point (x, y) for the pixel having the value of "1" after the reference image is differentiated and binarized, that is, the following formula Two
It is defined by the voting process for all points on the curve.

[Formula 2] ρ = xcos θ + ysin θ

The straight line extraction unit 11 first raster-scans an image to determine whether or not the pixel is a target pixel for Hough conversion, that is, whether or not the pixel being scanned has a value of "1". As a result of the determination, the voting process is executed on the pixel determined to be the target pixel. In the voting process, θ is 0
While changing from 1 to π, ρ for each θ is obtained from Equation 2, and the memory of the corresponding coordinates in the Hough space (θ-ρ space) is sequentially incremented by 1. The straight line extraction unit 11
The above processing is performed for all pixels, and thereafter, all the points on the Hough space where the number of votes becomes larger than a predetermined threshold value are extracted, and the θ-ρ value of these points is output as a linear parameter, End Hough transform.

Then, the straight line extraction unit 11 uses the extracted straight line as a parameter, for example, its slope (Δj / Δi,
Δ: increase amount) and j intercept (j coordinate value corresponding to i = 0 on a straight line).

In step 2, the interval calculator 12
Two straight lines L1 and L2 set as candidates for railway rails
The interval of the (two-dimensional straight line) in the real space is calculated. FIG. 6 is a flowchart showing a detailed procedure of the interval calculation process in step 2. First, in step 20, a pair of three-dimensional straight lines L1 ′ and L2 ′ in the real space corresponding to the specified two straight lines L1 and L2 are calculated. As a method of calculating this three-dimensional straight line, first, one straight line L1
Arbitrary two points p1 (i1, j1), p2 on the image in
From (i2, j2), the respective positions P1 (X1, Y1, Z1) and P2 (X in the real space corresponding to these two points p1 and p2
2, Y2, Z2) is specified. Where X in real space
The YZ coordinate system is set with reference to the mounting position of the stereo camera at the railroad crossing. The zenith direction at the mounting position is the Y axis, and the orthogonal coordinate system that defines a plane (ground) perpendicular to this Y axis is the Z coordinate system.
It is defined as an axis (distance direction from the cameras 2 and 3) and an X axis (direction orthogonal to the distance direction).

As a method of determining such arbitrary two points p1 and p2, for example, there are two points that divide the straight line L1 into three equal parts. The positions P1 and P2 are uniquely specified by the coordinates of p1 and p2 and the distance data using a well-known coordinate conversion formula. Then, from the specified two points P1 and P2, a straight line passing through these two points is specified as a three-dimensional straight line L1 '. Similarly, for the other straight line L2, a three-dimensional straight line L2 'is specified.

In step 21, this straight line L1 ',
The parallelism of L2 ′, that is, the degree of parallelism between the straight line L1 ′ and the straight line L2 ′ is calculated. This straight line L1 ', L2'
The parallelism of can be evaluated by the angle θ formed by the direction vectors of the straight lines L1 ′ and L2 ′. The angle θ formed by the direction vector can be uniquely calculated by a known geometric method.

Then, in step 22, it is judged whether or not the three-dimensional straight lines L1 ', L2' are parallel by comparing the angle .theta. Formed by the direction vector with a predetermined judgment value .theta.0. Whether or not the three-dimensional straight lines L1 and L2 are parallel can be determined by whether or not the calculated value of θ is smaller than the determination value θ0. When it is determined that they are parallel to each other, the distance D between the three-dimensional straight lines L1 ′ and L2 ′ is calculated in step 23. The three-dimensional straight lines L1 'and L2' have already been specified, and the distance D is specified by a known geometric method. On the other hand, if it is determined that they are not parallel to each other, in step 24, the straight line L
The interval between 1'and L2 'is output as "calculation impossible".

Then, in step 3, the object judging unit 13 judges the distance D and a judgment value D0 corresponding to the actual distance between the rails (for example, 1067 (mm) for narrow gauge, 143 for standard gauge).
5 (mm)) to determine whether or not the two straight lines L1 and L2 are rails. A method of comparing the interval D and the determination value D0 is, for example, a difference δ between D and D0 (δ = | D−
This is possible by specifying D0 |). Then, the determination as to whether or not the two straight lines L1 and L2 are rails is made based on whether or not the value of δ which is the comparison result is within the allowable range (for example, δ <10 (mm) If it is a railroad rail). In addition, in the process of step 24, when the distance D is output as "calculation impossible", the object determination unit 13 determines that the straight lines L1 and L2 are not rails.

When it is determined that the two straight lines L1 and L2 are rails, in step 4, the object determining unit 13
Recognizes the pair of straight lines L1 and L2 as railroad rails. Then, the object determination unit 13 specifies the position of the railroad rail on the image from the recognized inclinations of the pair of straight lines L1 and L2 and the j intercept. On the other hand, two straight lines L1 and L
When it is determined that 2 is not a railroad rail, in step 1, the straight line extraction unit 11 newly specifies two straight lines L1 and L2 from a plurality of straight line elements in different combinations. Then, until the object determination unit 13 recognizes the newly specified two straight lines L1 and L2 as railroad rails,
The above-mentioned steps 1 to 3 are repeatedly executed.

The straight line extracting section 11 and the interval calculating section 12
Is the two straight lines L for all combinations of straight line elements.
The interval between 1 and L2 may be calculated. In this case, the object determination unit 13 considers the above-mentioned interval from among all combinations of straight line elements, and considers the two most probable straight lines L1, L.
Recognize 2 as a rail.

Next, the monitoring unit 14 reads the reference image data corresponding to one frame and monitors an obstacle on the railroad track in the railroad crossing area based on the specified position of the railroad rail. Such an obstacle is specified by the shape of a vehicle or the like located on the recognized rail, that is, an aggregate of specific parallaxes. Alternatively, the obstacle is specified by matching a reference image stored in a memory (RAM or the like) (for example, a reference image in which there is no obstacle on the rail) with the captured reference image. Then, when it is determined that an alarm is necessary based on this monitoring information, the monitoring unit 14 uses an alarm device such as a monitor or a speaker to
It calls attention to passers-by, etc., or calls attention to a rail vehicle running on a rail (or controls (stops or slows down)).

The monitoring unit 14 can not only monitor pedestrians and vehicles passing through the railroad crossing area but also adjust the photographing areas of the cameras 2 and 3 based on the railroad crossing area thus set. The adjustment of the imaging area can be performed by specifying the relative positional deviation between the monitoring area on the image and the desired monitoring area based on the three-dimensional information of the rail rail specified from the image. Then, the monitoring unit 14 controls the angles and directions of the cameras 2 and 3 in order to change the monitoring area based on the specified positional deviation.

As described above, in the present embodiment, of the plurality of straight line elements existing in the image, the two straight lines L1 and L2 that are parallel to each other in the real space and the distance between them is the same as the rail width. It is judged as a rail. Thereby, even when there are a plurality of straight line elements, only two straight lines indicating the rail can be effectively recognized from the image. Further, according to this method, a pair of straight lines that are not parallel is eliminated during the processing, so that it is possible to achieve both high speed recognition processing capability and high recognition accuracy.

(Second Embodiment) FIG. 7 is a flow chart showing the detailed procedure of another interval calculation process in step 2. The railway rail recognition procedure according to the second embodiment is different from the procedure according to the first embodiment in the method of the interval calculation process, and if there are not many linear elements other than rails in the image in advance. This is an effective method when it is known. Hereinafter, the rail rail recognition procedure will be described, but the description of the procedure similar to the rail rail recognition procedure shown in FIG. 4 will be omitted here.

First, as a premise, the stereo camera picks up an image of a landscape including a monitoring area in which a linear locus is rarely present other than the railroad rail from diagonally above and outputs the picked-up image. In the present embodiment, it is important that the cameras 2 and 3 image the railway rail in an oblique direction in order to calculate an orthogonal vanishing point described later.

In step 1, the straight line extracting section 11
The reference image data corresponding to one frame is read, and the linear elements existing on the reference image plane, that is, the linear elements forming the rail are extracted. As shown in FIG. 8, the straight line extraction unit 11 identifies the extracted straight line elements as two straight lines L1 and L2 (two-dimensional straight lines) that are candidates for railroad rails.

Then, the straight line extraction unit 11 uses the extracted straight line as a parameter, for example, its slope (Δj / Δi,
Δ: increase amount) and j intercept (j coordinate value corresponding to i = 0 on a straight line).

In step 2, the interval calculator 12
The distance in the real space between the two straight lines L1 and L2 set as the railway rail candidates is calculated. Specifically, first,
In step 25, in the image plane, two straight lines L1,
The intersection of L2, that is, the vanishing point fa (ia, ja) regarding the two straight lines L1 and L2 is calculated. This vanishing point fa (ia,
ja) is a well-known geometric method, that is, two straight lines L.
It is specified by the common solution of 1 and L2 (the common solution of the linear equation specified from the slope and the j-intercept). The vanishing point fa is i
Although it is expressed by the −j coordinate system, it does not necessarily have to fit within the frame of the image (i = 0 to 511 and j = 0 to 199, respectively).

In the following step 26, two straight lines L1,
A vanishing point fb (ib, jb) of a straight line that is orthogonal to any of L2 in a three-dimensional space and is on a plane (ground surface) where these two straight lines L1 and L2 exist (hereinafter, referred to as “orthogonal vanishing point”). Will be calculated. In other words, the vanishing point regarding the straight line orthogonal to the pair of straight lines L1 and L2 is calculated as the orthogonal vanishing point fb based on the vanishing point fa. As shown in Expression 3, there is a correlation between the vanishing point fa and the orthogonal vanishing point fb, and if the vanishing point fa is specified, the orthogonal vanishing point fb is also uniquely specified. In the equation, constants PWV, P
WH is a vertical viewing angle and a horizontal viewing angle per pixel, respectively.

[Number 3] ib = - (1+ (PWV × ja) 2) / (PWH 2 × ia) jb = ja

In step 27, the orthogonal vanishing point fb
The intersection point p3 of the straight line R1 and the straight line L1 passing through (ib, jb)
(I3, j3) and an intersection point p4 (i4, j4) between the straight line R1 and the straight line L2 are set, respectively. A straight line R passing through the orthogonal vanishing point fb in the real space due to the characteristics of the vanishing point.
1 is always a straight line that can be orthogonal to the straight lines L1 and L2. That is, in the real space, the distance between these intersections p1 and p2 corresponds to the distance between the two straight lines L1 and L2. The straight line R1 may be set to any line as long as it passes through the orthogonal vanishing point fb (xb, yb), but in the present embodiment, a condition that it passes through the midpoint of one straight line L2 is added. Then, the straight line R1 is uniquely specified.

In step 28, the positions of the two intersections p3 and p4 in the real space are specified, and then the distance D in the real space between the two intersections p3 and p4 is calculated. The respective positions P3 (X3, Y3, Z3) in the real space corresponding to the intersection points p3, p4,
P4 (X4, Y4, Z4) is based on the well-known coordinate conversion formula,
It is uniquely specified by the distance data (i, j, d). Then, the distance D between the two straight lines L1 and L2 in the real space is specified by a known geometrical method, for example, the Pythagorean theorem, by the positions of the specified P3 and P4 in the real space. It

Then, in step 3, the object judging unit 13 determines the distance D and a judgment value D0 corresponding to the actual distance between the rails (for example, 1067 (mm) for narrow gauge, 143 for standard gauge).
5 (mm)) to determine whether or not the two straight lines L1 and L2 are rails. As described above, it is premised that there are not many linear elements other than the rails in the captured image. However, step 3
Even when the straight line extraction unit 11 extracts a straight line element other than the railroad rail (for example, a substantially straight line element, a shadow caused by the railroad rail), the object determination unit 13 eliminates the processing by It is possible to do so, which is preferable.

When it is determined that the two straight lines L1 and L2 are rails, the object determining unit 13 determines in step 4 the image from the recognized inclinations and the j intercepts of the pair of straight lines L1 and L2. Locate the rail rails above. On the other hand, when it is determined that the two straight lines L1 and L2 are not rails, the straight line extraction unit 11 again specifies the two straight lines L1 and L2 from the straight line elements in different combinations in step 1. Good. And
Two straight lines L1 and L2 newly identified by the object determination unit 13
Steps 1 to 3 described above until it is recognized as a railroad rail.
The process of is continuously repeated.

In the present embodiment, like the first embodiment, only two straight lines indicating the rails can be effectively recognized from the image, and in comparison with the first embodiment, the calculation processing amount is It has the effect of requiring less.

Although the preferred embodiments of the present invention have been described above, the present invention can be variously modified within the scope of the invention. For example, in the above-described embodiment, the object is a railway line (rail), but the present invention is not limited to this. The object may be a pair of straight lines arranged in parallel at a known interval, and for example, an example of the object is a white line drawn on a road (a survey line and an overtaking line).

The stereo camera has a function as a camera for outputting image data and a function as a distance sensor for outputting distance data in cooperation with the stereo image processing section 7. However, the present invention is not limited to a stereo camera, but a combination of a plurality of sensors having equivalent functions, for example, a set of a monocular camera and a distance sensor (for example, a laser radar such as a millimeter wave radar or a sound wave radar). May be The laser radar scans the surveillance area to output distance data showing a two-dimensional distribution of distances. Further, the distance data may be stored as a table in which the imaging range of the cameras 2 and 3 is scanned by the distance sensor in advance and the scan result is associated with the i-j coordinates on the image.

The straight line extracting method is not limited to the Hough transform. For example, the straight line extracting section 11 may be used for the main camera 2
It is also possible to detect the edge of the object based on the change in the brightness of the reference image obtained from, and recognize the straight line from the edge (for example, by the least square method).

Further, in the second embodiment, the object judging section 13 only compares the distance D between the two straight lines L1 and L2 in the real space with the judgment value D0, and the two straight lines L1 and L2 are compared. Parallelism (that is, whether parallel) is not verified. That is, the object determining unit 13 determines that the selected two straight lines L1 and L2 are included in an image including a plurality of straight line elements.
Even if the lines are not parallel, if the distance D is the same, it may be recognized as a rail. Therefore, the object determining unit 13 determines the two straight lines L recognized as the railroad rail.
You may verify the parallelism about 1 and L2. Specifically, the object determining unit 13 determines a new straight line (that is, the orthogonal vanishing point f that is orthogonal to the two straight lines L1 and L2 on the image plane).
It is also possible to define a straight line (R2) that passes through b and intersects L1 and L2 and is different from R1 and output an instruction to the interval calculation unit 12 to obtain the interval of the intersection with the straight line R2. That is, the two straight lines L1 and L2 specified in step 4 are returned to step 2, and the distance D is specified from the intersection different from p1 and p2. Then, by comparing the distance D with the determination value D0, the object determination unit 13 determines that the two straight lines L1 and L2.
However, it is possible to verify whether or not they are parallel.

[0049]

As described above, according to the present invention, the distance data corresponding to the image plane measured in the real space is used, and the distance between the pair of straight lines in the image plane is calculated in correspondence with the distance in the real space. is doing. With this, even in an image in which a plurality of straight line elements are present, it is possible to determine whether or not the two straight lines are the target object by comparing the calculated value with the value of the actual target object. Can recognize objects without misrecognition.

[Brief description of drawings]

FIG. 1 is a block configuration diagram of an object recognition device according to the present embodiment.

FIG. 2 is an explanatory diagram showing an image of one frame

FIG. 3 is an explanatory diagram of a pixel block set in a reference image.

FIG. 4 is a flowchart showing a procedure for recognizing a rail.

FIG. 5 is a diagram showing a screen for explaining a process of recognizing a rail rail.

FIG. 6 is a flowchart showing a detailed procedure of interval calculation processing.

FIG. 7 is a flowchart showing another calculation procedure in step 2 shown in FIG.

FIG. 8 is an explanatory diagram for explaining a processing step.

[Explanation of symbols]

1 Object recognition device 2 Main camera 3 sub camera 4 A / D converter 5 A / D converter 6 Image correction section 7 Stereo image processing unit 8 distance data memory 9 Image data memory 10 microprocessors 11 Straight line extraction unit 12 interval calculator 13 Object judgment section 14 Monitor

Continued front page    F term (reference) 2F065 AA04 AA06 AA22 AA23 AA32                       AA51 BB12 CC35 EE05 FF04                       FF05 GG04 JJ05 JJ08 JJ26                       QQ03 QQ24 QQ25 QQ32 QQ38                       RR07 SS09 SS15                 5B057 AA16 AA19 CA12 CB13 CD14                       CH01 CH11 DA12 DB03 DC03                 5C054 CC05 FC12 FC15 FD01 FF07                       HA05 HA19                 5L096 BA02 CA05 FA64 FA66 JA11                       LA05

Claims (7)

[Claims] 1. An object recognition apparatus for recognizing a pair of linear objects arranged in parallel at a known interval, which captures a scene including a monitoring area and outputs a captured image.
A camera, a distance sensor that outputs a two-dimensional distribution of distances in the monitoring area as distance data, and a straight line extraction unit that extracts a pair of two-dimensional straight lines that are candidates for the target object in the captured image plane. , Based on the distance data, to calculate a three-dimensional straight line in the real space corresponding to each of the two-dimensional straight line, to determine whether the pair of three-dimensional straight lines are parallel, the pair of tertiary If it is determined that the original straight lines are parallel,
A calculation unit that calculates the distance between the pair of three-dimensional straight lines, the distance calculated by the calculation unit, and a candidate value for the target object by comparing a determination value corresponding to the actual distance between the target object and An object recognizing device, comprising: an object determining unit that determines whether or not the pair of two-dimensional straight lines is the object. 2. An object recognition device for recognizing a pair of linear objects arranged in parallel at a known interval, wherein a first scene is captured, which includes a monitoring area, and a captured image is output.
A camera, a distance sensor that outputs a two-dimensional distribution of distances in the monitoring area as distance data, and a straight line extraction unit that extracts a pair of two-dimensional straight lines that are candidates for the target object in the captured image plane. , In the captured image plane, calculates a vanishing point for the pair of two-dimensional straight lines, based on the vanishing point, calculates a vanishing point for a two-dimensional straight line orthogonal to the pair of two-dimensional straight lines as an orthogonal vanishing point, A two-dimensional straight line passing through the orthogonal vanishing point is calculated as a first intersection and a second intersection that intersect with each of the pair of two-dimensional straight lines, and based on the distance data, the first intersection and By comparing a calculation unit that calculates a space in the real space with the second intersection, a space calculated by the calculation unit, and a determination value corresponding to the actual space of the object, An object recognition device, comprising: an object determination unit that determines whether the pair of two-dimensional straight lines that are candidates for the object are the objects. 3. The object recognition device according to claim 2, wherein the first camera images the pair of linear objects in an oblique direction. 4. A stereo camera which captures a landscape including a surveillance area and outputs a captured image, wherein the distance sensor is a stereo composed of the first camera and the second camera. The object recognition device according to claim 1, further comprising a camera. 5. The object recognition device according to any one of claims 1 to 4, wherein the object is a rail of a railroad track. 6. A pair of straight lines arranged in parallel at known intervals based on a captured image of a landscape including a surveillance region and distance data showing a two-dimensional distribution of distances in the surveillance region. In the object recognition method for recognizing an object, a step of extracting a pair of two-dimensional straight lines that are candidates for the object in a captured image plane, and an actual method corresponding to each of the two-dimensional straight lines based on distance data. A step of calculating a three-dimensional straight line in space, a step of determining whether the pair of three-dimensional straight lines are parallel, if it is determined that the pair of three-dimensional straight lines are parallel, The step of calculating the interval of the three-dimensional straight line, the calculated interval, by comparing the determination value corresponding to the actual interval of the object, the pair of two-dimensional straight lines that are candidates for the object Object recognition method characterized by a step of determining whether or not the object. 7. A pair of straight lines arranged in parallel at a known interval based on a captured image of a landscape including a surveillance region and distance data showing a two-dimensional distribution of distances in the surveillance region. In the object recognition method for recognizing an object, a step of extracting a pair of two-dimensional straight lines that are candidates for the object in a captured image plane, and a vanishing point regarding the pair of two-dimensional straight lines in the captured image plane. A step of calculating, based on the vanishing point, a step of calculating a vanishing point regarding a two-dimensional straight line orthogonal to the pair of two-dimensional straight lines as an orthogonal vanishing point, and a two-dimensional straight line passing through the orthogonal vanishing point is the pair. Calculating a point intersecting each of the two-dimensional straight lines as a first intersection and a second intersection, and the first intersection and the second intersection based on the distance data.
And a step of calculating an interval in the real space between the intersection and the intersection, and by comparing the calculated interval with a determination value corresponding to the actual interval of the object, the object becomes a candidate. Determining whether or not the pair of two-dimensional straight lines is the target object.