JP4150223B2 - Level crossing monitoring device and level crossing monitoring method - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a railroad crossing monitoring apparatus and a railroad crossing monitoring method for monitoring a situation in a railroad railroad crossing.
[0002]
[Prior art]
In recent years, monitoring devices that monitor the situation in a predetermined monitoring area have attracted attention and practical application. This monitoring device is mounted on a moving body such as a vehicle or an aircraft, or is fixed to a stationary body such as a support. With regard to the railroad crossing monitoring apparatus as an example of the latter, conventionally, a monorail camera is used to image railroad crossings including railroad rails (hereinafter referred to as “railroad rails”), and the position of the railroad rails in the captured image plane is specified. Then, with reference to a preset level crossing area, it is monitored whether or not an obstacle such as an automobile or a person has entered the level crossing area. For example, Japanese Patent Application Laid-Open No. 10-341427 discloses a monitoring device in which two monocular cameras are installed diagonally across a railroad crossing through a railroad rail, and the entire range of the crossing is monitored over a wide range.
[0003]
[Problems to be solved by the invention]
Such a crossing monitoring device classifies an object to be monitored or not to be monitored depending on whether or not the object projected in the monitoring area has entered the crossing area. Therefore, as a premise for performing such classification, it is necessary to specify a crossing area in real space. However, conventionally, this must be initialized manually, and it is complicated to perform this operation every time the apparatus is installed. In addition, when the level crossing area is initially set by such manual work, the initially set level crossing area and the actual level crossing do not correspond in position due to misalignment caused by changes in the camera over time. It becomes impossible to perform effective monitoring. On the other hand, it is conceivable to recognize a railroad rail from the image plane and set a monitoring area therefrom, but it is difficult to accurately recognize the railroad rail with a monitoring device using only a monocular camera.
[0004]
In this prior art, since information from two monocular cameras is integrated and processed, it is necessary to share the information of the two cameras. However, the two cameras arranged diagonally are electrically connected. In order to establish a connection, it is necessary to straddle at least railway rails. Installation of such a monitoring device on existing railroad rails causes problems such as an increase in construction scale and a large economic burden as well as a burden on maintenance work. .
[0005]
The present invention has been made in view of such circumstances, and an object thereof is to provide a crossing monitoring device and a crossing monitoring method for setting a crossing area in a monitoring area by effectively recognizing a railroad rail in an image plane. It is to be.
[0006]
Another object of the present invention is to provide a crossing monitoring device and a crossing monitoring method that reduce the complexity of installation work and maintenance work, and monitor a wide range with a plurality of cameras.
[0007]
[Means for Solving the Problems]
  In order to solve this problem, a first invention is a crossing monitoring device that monitors a situation in a railroad crossing, and includes a stereo camera, a stereo image processing unit, a specifying unit, and a setting unit. Providing equipment. In such a crossing monitoring device,The stereo camera captures a scene including the monitoring area and outputs a pair of image data. The stereo image processing unit calculates parallax by stereo matching based on a pair of image data, and a parallax group related to image data corresponding to one frame corresponds to a coordinate position on an image plane defined by the image data. The attached distance data is output. The specifying unit specifies a railroad rail on an image plane defined by the image data. The setting unit sets a railroad crossing area for classifying an object projected in the monitoring area into one that should be monitored and one that does not need to be monitored, based on the identified railroad rail. Here, the specifying unit extracts a pair of two-dimensional straight lines that are candidates for railroad rails on the image plane, and calculates a three-dimensional straight line in real space corresponding to each of the two-dimensional straight lines based on the distance data. Then, it is determined whether or not the pair of three-dimensional straight lines are parallel. In addition, when the specific unit determines that the pair of three-dimensional straight lines are parallel, the specific unit calculates a distance between the pair of three-dimensional straight lines, and a determination corresponding to the calculated distance and the actual distance between the rails. By comparing the values, it is determined whether or not the pair of two-dimensional straight lines that are candidates for the rail is a rail.
[0008]
Here, in the first invention, it is preferable to further include an object recognition unit and a monitoring unit in addition to the above-described configuration. The object recognition unit recognizes the object in the monitoring area based on the distance data. The monitoring unit determines whether or not the recognized object exists in the crossing area, and monitors the recognized object when determining that the object exists in the crossing area.
[0014]
  According to a second aspect of the invention, image data obtained by capturing a landscape including a monitoring area with a stereo camera, and coordinates on an image plane that are output from a stereo image processing unit and defined by a parallax group and image data related to image data corresponding to one frame There is provided a railroad crossing monitoring method for monitoring a situation in a railroad railroad crossing based on distance data associated with a position. In this monitoring method, in the image plane defined by the image data output from the stereo camera, the specifying unit extracts a pair of two-dimensional straight lines that are candidates for the rail, and the specifying unit is a stereo image processing unit. A step of calculating a three-dimensional straight line in real space corresponding to each of the two-dimensional straight lines based on the distance data output from the step, and a step of determining whether or not the pair of three-dimensional straight lines are parallel by the specifying unit And when the specific unit determines that the pair of three-dimensional straight lines are parallel, the step of calculating the distance between the pair of three-dimensional straight lines, A pair of two-dimensional lines is determined by the step of determining whether the pair of two-dimensional straight lines that are candidates for the railroad rail is a railroad rail by comparing with a determination value corresponding to the interval of When it is determined that the line is a railroad rail, the setting unit determines whether the object projected in the monitoring area is to be monitored or not to be monitored based on the pair of two-dimensional straight lines. And setting a crossing area for classification.
[0015]
  Here, in the second invention, the object recognizing unit recognizes the object in the monitoring area based on the distance data, and the monitoring unit determines whether the recognized object exists in the crossing area. And a step of monitoring the recognized target object when the monitoring unit determines that the target object is present in the cut area.
[0020]
DETAILED DESCRIPTION OF THE INVENTION
FIG. 1 is a block configuration diagram of a railroad crossing monitoring apparatus according to the present embodiment. This level crossing monitoring apparatus includes a first object recognition unit 1, a second object recognition unit 1a, and a monitoring unit 15, and an object entering the level crossing area projected in the monitoring area (passage) Monitor cars, passers-by, or obstacles). The first object recognition unit 1 recognizes a railroad rail necessary for setting a railroad crossing region and recognizes an object to be monitored using the captured image. Similarly to the first object recognition unit 1, the second object recognition unit 1a also recognizes the railway rail and the object using the captured image. In these object recognition units 1 and 1a, the constituent elements (2a to 14a) in which alphabets are added to the numbers are the constituent elements (2 to 14) of the reference numerals of the numerals without alphabets in the corresponding numerals. 14). Hereinafter, in the specification, only the first object recognition unit 1 will be described, but the same applies to the second object recognition unit 1a unless otherwise specified. That is, the second object recognition unit 1a can be understood by replacing the reference numerals of the first object recognition unit 1 described below with reference numerals with alphabets.
[0021]
In the first object recognition unit 1, the stereo camera 2 is fixed to a post or the like in the vicinity of the level crossing, and as shown in FIG. 2, the crossing including the monitoring area, that is, the railroad rail, is imaged obliquely from above. . As shown in FIG. 2, in addition to the railway rail that is the object to be detected, a straight road, a horizon, and the like are also projected on the image plane, and thus there are many linear elements.
[0022]
The stereo camera 2 has a pair of cameras 3 and 4 arranged at a predetermined interval (camera base line length). Each camera 3 and 4 includes an image sensor such as a CCD or a CMOS sensor. Yes. The main camera 3 captures a reference image (right image) necessary for performing stereo image processing, and the sub camera 4 captures a comparison image (left image). In a state where each other is synchronized, each analog image output from the cameras 3 and 4 is converted into a digital image of a predetermined luminance gradation (for example, 256 gradation gray scale) by the A / D converters 5 and 6. Is converted to The digitized image is subjected to brightness correction, image geometric conversion, and the like in the image correction unit 7. Usually, there is an error in the mounting positions of the pair of cameras 3 and 4 to some extent, but there is a shift in the left and right images due to the error. In order to correct this deviation, geometrical transformation such as image rotation or translation is performed using affine transformation or the like.
[0023]
Through such image processing, reference image data is obtained from the main camera 3, and comparison image data is obtained from the sub camera 4. These image data are a set of luminance values (0 to 255) of each pixel included in the individually set effective image area. In the ij coordinate system on the image plane defined by the image data, the lower left corner of the image is the origin, the horizontal direction is the i coordinate axis, and the vertical direction is the j coordinate axis. Both image data corresponding to one frame, which is the minimum display unit of one image, are stored in the image data memory 9 as needed.
[0024]
The stereo image processing unit 8 calculates distance data for a captured image corresponding to one frame based on the reference image data and the comparison image data. Here, “distance data” is a set of parallax d calculated from a captured image corresponding to one frame, and each parallax d is associated with a position (i, j) on the image plane. Each parallax d is calculated for each pixel block of a predetermined area (for example, 4 × 4 pixels) constituting the reference image. FIG. 3 is an explanatory diagram of pixel blocks set in the reference image. A reference image area of pixels is divided into a matrix by a pixel block PBij of 4 × 4 pixels. A parallax group corresponding to the number of pixel blocks PBij is calculated from the captured image corresponding to one frame. As is well known, the parallax d is a horizontal shift amount related to the pixel block PBij that is the calculation target, and has a large correlation with the distance to the target object projected on the pixel block PBij. In other words, the closer the object projected in the pixel block PBij is to the cameras 3 and 4, the larger the parallax d of the pixel block PBij is, and the farther the object is, the smaller the parallax d is (if the object is infinitely far away, the parallax d becomes 0).
[0025]
When calculating the parallax d regarding a certain pixel block PBij (correlation source), a region (correlation destination) having a correlation with the luminance characteristic of the pixel block PBij is specified in the comparison image. As described above, the distance from the cameras 3 and 4 to the object appears as a horizontal shift amount between the reference image and the comparison image. Therefore, when searching for the correlation destination in the comparison image, it is only necessary to search on the same horizontal line (epipolar line) as the j coordinate of the pixel block Pij as the correlation source. The stereo image processing unit 8 shifts the correlation between the correlation source and the correlation destination candidate while shifting the epipolar line one pixel at a time within a predetermined search range set based on the i coordinate of the correlation source. Sequential evaluation (stereo matching). In principle, the amount of horizontal deviation of the correlation destination (one of the correlation destination candidates) determined to have the highest correlation is defined as the parallax d of the pixel block PBij.
[0026]
The correlation between two pixel blocks can be evaluated, for example, by calculating a city block distance CB. Formula 1 shows the basic form of the city block distance CB. In the equation, p1ij is the luminance value of the ijth pixel of one pixel block, and p2ij is the ijth luminance value of the other pixel block. The city block distance CB is the sum of the difference (absolute value) between the luminance values p1ij and p2ij corresponding to each other in the entire pixel block, and the smaller the difference is, the greater the correlation between the two pixel blocks is. Yes.
[Expression 1]
CB = Σ | p1ij−p2ij |
[0027]
Basically, among the city block distances CB calculated for each pixel block existing on the epipolar line, the pixel block having the smallest value is determined as the correlation destination. The amount of deviation between the correlation destination specified in this way and the correlation source is the parallax d. The hardware configuration of the stereo image processing unit 8 for calculating the city block distance CB is disclosed in Japanese Patent Laid-Open No. 5-1114099, and should be referred to if necessary. The distance data calculated through such processing is stored in the distance data memory 10 as needed.
[0028]
The microcomputer 11 includes a CPU, a ROM, a RAM, an input / output interface, and the like. When the microcomputer 11 is functionally grasped, the microcomputer 11 includes a specifying unit 12, a setting unit 13, and an object recognition unit 14. The specifying unit 12 reads reference image data (hereinafter simply referred to as “image data”) P from the image data memory 9, and specifies a railroad rail on an image plane defined by the image data P. Then, the specified railroad rail is output to the setting unit 13. The setting unit 13 sets a railroad crossing area A in the monitoring area based on the identified railroad rail. Here, the “crossing area” means an area set to classify an object projected in the monitoring area into one that should be monitored and one that does not need to be monitored. Identified. In the present embodiment, the level crossing area A is uniquely set by the setting unit 13 and output to the monitoring unit 15. On the other hand, the object recognition unit 14 reads the distance data D and recognizes the object O in the monitoring area based on the distance data D. The recognized object O is output to the monitoring unit 15. In this specification, in order to avoid confusion with a level crossing area Aa, which will be described later, this level crossing area A is referred to as a “first level crossing area A”. Similarly, in order to avoid confusion with the distance data Da, the distance data D is referred to as “first distance data D”.
[0029]
Here, the positional relationship between the first object recognition unit 1 and the second object recognition unit 1a will be described. The stereo camera 2 on the first object recognition unit 1 side and the stereo camera 2a on the second object recognition unit 1a side are provided on the same side with the railroad rail as a reference. Specifically, one stereo camera 2a is preferably arranged in the vicinity of the other stereo camera 2, and as shown in FIG. 4, it is more preferable that these are arranged together in one place. . However, in this invention, each stereo camera 2 and 2a should just be provided in the same side on the basis of a railroad rail, and is not limited to the arrangement | positioning mentioned above. However, as shown in the present embodiment, it is preferable to arrange these in one place in order to reduce the size of the installation work. In the figure, one stereo camera 2a is coincident with the other stereo camera 2 in the center position (the middle point on the camera base line length) and is spaced upward by Δy. In addition, one stereo camera 2a is arranged such that its visual axis is inclined to the left side by an angle θ with respect to the other stereo camera 2. In such an attached state, each of the stereo cameras 2 and 2a captures only a part of the scenery in the monitoring area. However, by cooperating with each other, all the scenery in the monitoring area is covered. Therefore, the stereo cameras 2 and 2a do not need to widen the angle of view of each of the wide monitoring areas, so that the image is monitored while suppressing the occurrence of image distortion and maintaining high resolution. can do.
[0030]
In the microcomputer 11a on the second object recognition unit 1a side, the specifying unit 12a reads the image data Pa from the image data memory 9a, and specifies the railroad rail on the image plane defined by the image data Pa. . Then, the setting unit 13a sets a railroad crossing area Aa (hereinafter referred to as “second railroad crossing area Aa”) in the monitoring area based on the identified railroad rail, and the second railroad crossing area Aa. Is output to the monitoring unit 15. On the other hand, the object recognition unit 14a reads the distance data Da (hereinafter referred to as “second distance data Da”), and based on the read second distance data Da, the object recognition unit Oa in the monitoring area. In addition to performing recognition, the recognized object Oa is output to the monitoring unit 15.
[0031]
Here, the microcomputer 11 on the first object recognition unit 1 side uses the first distance data D to specify the first three-dimensional coordinates for specifying the position in the real space by a known coordinate conversion formula. The system is defined. The first three-dimensional coordinate system uses the position of the stereo camera 2 as a reference, the origin is directly below the center of the stereo camera 2 (the ground), the zenith direction is the y1 axis, and the orthogonality defines a plane perpendicular to the y1 axis. Let the coordinate system be the x1 axis and the z1 axis. At this time, the z1 axis coincides with the viewing direction of the cameras 3 and 4. Further, the microcomputer 11a on the second object recognition unit 1a side uses a second three-dimensional coordinate system for specifying a position in the real space by a known coordinate conversion formula based on the first distance data D. Is stipulated. This second three-dimensional coordinate system is based on the position of the stereo camera 2a, with the origin directly below the center of the stereo camera 2a (the ground), the zenith direction as the y2 axis, and an orthogonality defining a plane perpendicular to the y2 axis. Let the coordinate system be the x2 axis and the z2 axis. At this time, the z2 axis coincides with the viewing direction of the cameras 3a and 4a.
[0032]
Referring to FIG. 1 again, the monitoring unit 15 recognizes the object O recognized by the object recognition unit 14 on the first object recognition unit 1 side and the object recognition on the second object recognition unit 1a side. Information with the object Oa recognized by the unit 14a is integrated. And the object which exists in a level crossing is monitored. At this time, since the monitoring unit 15 integrates information on the object O and the object Oa recognized in the independent three-dimensional coordinate system, at least one of the first and second three-dimensional coordinate systems is used. A third three-dimensional coordinate system different from is defined. In other words, the coordinate position of the object recognized by one object recognition unit 14 is coordinate-transformed from the first three-dimensional coordinate system to the third three-dimensional coordinate system, and the other object recognition unit 14a. The coordinate position of the object recognized by is coordinate-transformed from the second three-dimensional coordinate system to the third three-dimensional coordinate system. Here, each of the objects O and Oa coordinate-converted into the third three-dimensional coordinate system is referred to as an object Oo. Then, based on the third three-dimensional coordinate system, a third level crossing area including the first level crossing area A and the second level crossing area Aa is set, and the object Oo exists in the third level crossing area. If so, this object Oo is monitored.
[0033]
FIG. 5 is a flowchart showing a procedure for setting a crossing area. This routine is called at predetermined intervals and is executed by each of the microcomputers 11 and 11a. Hereinafter, in order to avoid duplication description, the procedure for setting the first level crossing area A in the microcomputer 11 on the first object recognition unit 1 side will be described unless otherwise specified.
[0034]
First, in step 1, the specifying unit 12 reads the image data P corresponding to one frame, and extracts a linear element existing on the image plane defined by the image data P. Thereby, all the linear elements which comprise a railroad rail, a road, and a horizon are extracted. Then, as shown in FIG. 6, a pair of two-dimensional straight lines L1 and L2 that are candidates for railroad rails are identified by an arbitrary combination from the extracted linear elements. The specified pair of two-dimensional straight lines L1 and L2 are respectively defined by parameters such as slope (Δj / Δi, Δ: increase amount) and j intercept (j coordinate value corresponding to i = 0 on the straight line). .
[0035]
Here, the straight line element extraction method includes a known straight line extraction method, for example, a method such as image differentiation / binarization processing, Hough transform (Hough transform), template matching, and the like. The Hough transform as an example is based on the logic that “a straight line can be uniquely determined by using the distance (ρ) from the origin and the inclination (θ) viewed from the X axis as parameters”. , Ρ, θ conversion into a Hough space (parameter space). This Hough transform is performed on all the coordinate points on the Hough space indicating all straight lines passing through the point (x, y) for pixels having a value of “1” after differentiation and binarization of the reference image, That is, it is defined by the voting process for all points on the curve shown in Equation 2 below.
[Expression 2]
ρ = xcosθ + ysinθ
[0036]
The specifying unit 12 first raster-scans the image sequentially to determine whether or not the pixel is a Hough transform target pixel, that is, whether or not the pixel being scanned has a value of “1”. As a result of the determination, the voting process is executed for the pixel determined to be the target pixel. In the process of voting, while changing θ from 0 to π, ρ for each θ is obtained from Equation 2, and the memory of the corresponding coordinate in the Hough space (θ-ρ space) is sequentially incremented by one. The above processing is performed for all the pixels, and thereafter, all points on the Hough space where the number of votes is greater than a predetermined threshold are extracted, and the θ-ρ values of these points are output as linear parameters. The Then, the specifying unit 12 ends the Hough conversion.
[0037]
In step 2, the specifying unit 12 calculates an interval in the real space between the pair of two-dimensional straight lines L1 and L2 specified as railway rail candidates. FIG. 7 is a flowchart showing a detailed procedure of the interval calculation process in step 2 shown in FIG. First, in step 20, a pair of three-dimensional straight lines L1 'and L2' on the real space corresponding to the pair of two-dimensional straight lines L1 and L2 are calculated. In order to calculate these, first, two arbitrary points p1 (i1, j1) and p2 (i2, j2) on the image in one two-dimensional straight line L1 are specified, and these two points p1 and p2 are associated. The positions P1 (xp1, yp1, zp1) and P2 (xp2, yp2, zp2) in the real space (first three-dimensional coordinate system) are specified. A straight line passing through the two specified points P1 and P2 is calculated as a three-dimensional straight line L1 '. Here, as two arbitrary points on the image in the two-dimensional straight line L1, for example, there are points that divide the two-dimensional straight line L1 into three equal parts. Further, the respective positions P1 and P2 in the real space are uniquely specified by the coordinate positions of p1 and p2 on the image and the distance D data corresponding thereto using a well-known coordinate conversion formula. A three-dimensional straight line L2 'is specified for the other two-dimensional straight line L2 by the same method.
[0038]
In step 21, the parallelism between the pair of three-dimensional straight lines L1 'and L2', that is, the degree of parallelism between the three-dimensional straight line L1 'and the three-dimensional straight line L2' is calculated. The parallelism of the three-dimensional straight lines L1 'and L2' is evaluated by the angle θ formed by the direction vectors of the three-dimensional straight lines L1 'and L2'.
[0039]
In step 22, it is determined whether or not the three-dimensional straight lines L1 'and L2' are parallel by comparing the angle θ formed by the direction vector with a predetermined determination value θ0. In the present embodiment, when the calculated value of θ is smaller than the determination value θ0, it is determined that these three-dimensional straight lines L1 'and L2' are parallel. If an affirmative determination is made in step 22, that is, if it is determined that they are parallel, the process proceeds to step 23, and the interval D between the three-dimensional straight lines L 1 ′ and L 2 ′ is calculated. These three-dimensional straight lines L1 'and L2' have already been specified, and the distance D is specified by a well-known geometric method. On the other hand, if a negative determination is made in this step, that is, if it is determined that they are not parallel, the process proceeds to step 24, and the interval between the three-dimensional straight lines L1 'and L2' is output as "uncalculated".
[0040]
Then, in step 3 following step 2, the specifying unit 12 determines that the interval D and a determination value D0 corresponding to the actual interval of the rail (for example, 1067 (mm) for narrow gauge, 1435 (mm) for standard gauge). Are compared to determine whether these two-dimensional two straight lines L1 and L2 are railway rails. In the present embodiment, for example, if the value of the difference δ (D = | D−D0 |) between D and D0 is within an allowable range, it is determined that the two-dimensional straight lines L1 and L2 are railway rails ( For example, if δ <10 (mm), it is a railroad rail). In the process of step 24, when the interval D is output as “uncalculated”, the specifying unit 12 automatically determines that the straight lines L1 and L2 at this time are not railroad rails.
[0041]
When an affirmative determination is made in step 3, the specifying unit 12 determines that the two-dimensional straight lines L1 and L2 are railroad rails, and recognizes the two-dimensional straight lines L1 and L2 as railroad rails (step 4). In the image plane, the rails are defined by parameters (for example, the inclination and j-intercept or the coordinate positions of the ends of the lines L1 and L2 on the image) that define these two-dimensional straight lines L1 and L2. The
[0042]
On the other hand, when a negative determination is made in step 3, the specifying unit 12 determines that these two-dimensional straight lines L1 and L2 are not railroad rails, and returns to step 1 to newly add different combinations of two-dimensional straight lines L1 and L2. To be specific. Then, in step 3, the processes in steps 1 to 3 described above are repeatedly executed a predetermined number of times corresponding to the combination of the extracted linear elements until an affirmative determination is made. Even if this process is repeated a predetermined number of times, if there are no straight lines L1 and L2 that are determined to be railroad rails, the specifying unit 12 determines that the image is inappropriate and determines that the image is inappropriate. It is preferable to move to the processing target frame.
[0043]
Note that after the rail is recognized in step 4, the specifying unit 12 may repeatedly execute the above-described routine a plurality of times for the remaining linear elements in the image data P to be processed. As a result, as shown in FIG. 4, even if the railway rails to be recognized are double tracks (or more than double tracks), the specifying unit 12 recognizes these rail rails without overlooking and confusing them. be able to.
[0044]
In step 5, the setting unit 13 sets the first level crossing area A based on the identified railroad rail. Hereinafter, an exemplary procedure for setting a crossing area will be described. First, image data P is read from the image data memory 9. Then, the recognized railroad rail is specified on the image plane defined by the read image data P. Then, by obtaining a pair of three-dimensional straight lines in the real space corresponding to the railroad rail, the position of the railroad rail in the real space (first three-dimensional coordinate system) is specified. Then, as shown in FIG. 8, first, on the same plane as this railroad rail, a plane area (in which a width W is added to the outside of the railroad rail and the distance L is considered in the rail rail extending direction) In the figure, a hatched area is set. The first level crossing area A as a three-dimensional space is set by taking into account the height direction (not shown) in consideration of the vehicle height of the passing vehicle and the height of the passer-by in this plane area. The
[0045]
Here, in the setting procedure described above, “the same plane as the railroad rail” can be uniquely specified as a common plane of a pair of three-dimensional straight lines. In addition, the “planar region” first adds an arbitrary width W in the orthogonal direction with reference to the pair of three-dimensional lines on the same plane as the pair of three-dimensional lines. Then, by setting the distance L determined to sufficiently cover the length of the level crossing in the extending direction of the three-dimensional straight line with reference to the position where the level crossing will exist, this plane area is specified. Is done. In addition, when there are a plurality of railway rails in the monitoring area, the setting unit 13 pays attention only to the outermost rail pair and sets all the areas inside them as the first railroad crossing area A. It is preferable to do. Thereby, the first level crossing area A is set so as to cover the entire level crossing.
[0046]
In the present embodiment, as shown in FIG. 9, the setting unit 13 on the first object recognition unit 1 side performs the first railroad crossing according to the above-described procedure based on a plurality of recognized railroad rails (double lines). Region A is set (the height direction is not shown). However, in practice, the recognizable area is narrowed according to the field angle range of the stereo camera 2, and thus the planar area is substantially trapezoidal. And the 1st level crossing area | region A is set by considering a height direction to this plane area | region. The setting unit 13 defines the first level crossing area A based on parameters that define the space (such as a representative coordinate position that defines the first level crossing area A). Similarly, the setting unit 13a on the second object recognition unit 1a side also sets the second level crossing area Aa based on the recognized double-track railway rail (height direction) as shown in FIG. Is not shown).
[0047]
On the other hand, the object recognition unit 14 (the first object recognition unit 1 side) reads the first distance data D from the distance data memory 10 and recognizes the object O based on the first distance data D. Do. For example, the object recognition unit 14 divides the first distance data D in a grid pattern at a predetermined interval, extracts the data of the object O for each division, and creates a histogram. Then, the existence position of the object O representing each section and the distance thereof are obtained from this histogram. The distances for each section are sequentially compared from the left to the right of the image, and those whose distances in the front-rear direction and the horizontal direction are close to each other are collected as a group. The data arrangement direction is checked for each group, and the group is divided at a portion where the arrangement direction greatly changes, and each group is recognized as the object O from the arrangement direction of the first distance data D. Then, the object O is specified by parameters defining the group (average distance, (left and right) end positions, etc.). Similarly, the object recognition unit 14a (second object recognition unit 1a side) recognizes the object Oa.
[0048]
FIG. 10 is a flowchart showing a procedure for monitoring a crossing area. In step 1, the monitoring unit 15 acquires the first level crossing area A and the second level crossing area Aa, converts the coordinates into the third three-dimensional coordinate system, and sets the third level crossing area. For example, the monitoring unit 15 defines the third three-dimensional coordinate system with the origin directly below the center of the stereo cameras 2 and 2a. The zenith direction is the y3 axis, and the orthogonal coordinate system defining a plane perpendicular to the y3 axis is the x3 axis and the z3 axis. Here, the x3 axis coincides with the bisecting direction of the x1 axis and the x2 axis, and the z3 axis coincides with the bisecting direction of the z1 axis and the z2 axis. In the third three-dimensional coordinate system, the monitoring unit 15 performs coordinate conversion of the first level crossing area A into the third three-dimensional coordinate system based on Expression 3 shown below, and the second level crossing area based on Expression 4 Aa is coordinate-transformed into the third three-dimensional coordinate system.
[Equation 3]
x3 = x1cos (-θ / 2) -z1sin (-θ / 2)
y3 = y1
z3 = x1sin (-θ / 2) + z1cos (-θ / 2)
[0049]
[Expression 4]
x3 = x2cos (θ / 2) −z2sin (θ / 2)
y3 = y2
z3 = x2sin (θ / 2) + z2cos (θ / 2)
[0050]
In the third three-dimensional coordinate system, an area including the first level crossing area A and the second level crossing area Aa is set as a third level crossing area (a rectangular area indicated by a broken line).
[0051]
Next, in Step 2, the monitoring unit 15 determines whether or not the object O or the object Oa is output from either one of the object recognition units 14 and 14a. When an affirmative determination is made in step 2, the monitoring unit 15 acquires this. The monitoring unit 15 knows from which of the object recognition units 14 and 14a data is acquired, and based on this, the acquired object is expressed in the third three-dimensional form using Equation 3 or Equation 4. The coordinates are converted into the coordinate system (step 3). By performing this process, the monitoring unit 15 can integrate the information on the objects O and Oa recognized by the object recognition units 14 and 14a and understand the position of the object Oo existing in the crossing. . On the other hand, if a negative determination is made in step 2, the monitoring unit 15 determines that there is no object in the monitoring area, and exits this routine.
[0052]
In step 4 following step 3, the monitoring unit 15 determines whether or not the recognized object Oo exists in the third level crossing area. If an affirmative determination is made in this step, in step 5, the monitoring unit 15 determines that the object Oo exists in the third level crossing area to be monitored, and if it is determined that an alarm is necessary, a monitor, a speaker, or the like Alerts passers-by with a warning device. Further, the monitoring unit 15 may alert the railway vehicle traveling on the railroad rail, or may control (stop or decelerate) the railcar. On the other hand, if a negative determination is made in this step, the monitoring unit 15 determines that the object Oo does not exist in the third level crossing area, and exits this routine.
[0053]
As described above, in this embodiment, even if two stereo cameras are used, since they are set together in one place, the construction work is reduced in scale and the burden of maintenance work is reduced. Can be reduced. Further, since the position of the railroad rail is detected from the image, and the monitoring area is automatically set based on the detected railroad rail, the trouble of setting and inputting the monitoring area can be saved. Further, even if a device misalignment or the like occurs, the monitoring area is determined based on the railroad rail, so that the monitoring area is always properly maintained and contributes to improving the reliability of the crossing monitoring device.
[0054]
In the present embodiment, two stereo cameras, that is, two object recognition units are provided. However, a crossing monitoring apparatus including more object recognition units has the same effect.
[0055]
The monitoring unit 15 not only monitors passers-by and vehicles passing through the third level crossing area, but also adjusts the shooting areas of the stereo cameras 2 and 2a based on the set third level crossing area. You can also. For example, the monitoring unit 15 changes the direction of the stereo cameras 2 and 2a based on the third level crossing area specified from the railroad rail so that the third level crossing area is located at the center of the imaging area of the stereo camera. Etc. Thereby, the burden of maintenance work can be further reduced.
[0056]
In the present embodiment, the microcomputers 11 and 11a perform processing by integrating them in the monitoring unit 15 in consideration of independent three-dimensional coordinate systems. However, the microcomputers 11 and 11a may execute the respective processes in consideration of a common three-dimensional coordinate system. In this case, since the monitoring part 15 can integrate the information regarding the target object, without converting the coordinate of the acquired target object, it is preferable.
[0057]
In addition, as a method for setting the level crossing area, a plane in which a predetermined margin is added to the recognized railroad rail end in the image plane is specified, this plane is developed in real space, and the height direction is added thereto. Thus, a crossing area may be set.
[0058]
Furthermore, in the present embodiment, the arrangement of the stereo cameras 2 and 2a is exemplary, and all the scenery including the monitoring area is provided by the cooperation of the cameras provided on the same side with the railroad rail as a boundary. Can be captured. Accordingly, the setting position of the third three-dimensional coordinate system set by the monitoring unit 15 is not limited to the above description.
[0059]
【The invention's effect】
As described above, in the present invention, the railroad rail can be accurately recognized by taking the distance data into consideration in the captured image. Therefore, the railroad crossing region can be set based on the recognized railroad rail. . As a result, the level crossing area is properly maintained on the level crossing, so that the reliability in monitoring accuracy can be improved.
[Brief description of the drawings]
FIG. 1 is a block diagram of a railroad crossing monitoring apparatus according to an embodiment.
FIG. 2 is an explanatory diagram showing an image of one frame.
FIG. 3 is an explanatory diagram of a pixel block set in a reference image.
FIG. 4 is an explanatory diagram showing the position of the stereo camera.
FIG. 5 is a flowchart showing a procedure for setting a crossing area.
FIG. 6 is a diagram showing a screen for explaining a process of recognizing a railroad rail.
7 is a flowchart showing a detailed procedure of interval calculation processing in step 2 shown in FIG. 5;
FIG. 8 is an explanatory diagram showing a crossing area.
FIG. 9 is an explanatory diagram showing another example of a crossing area.
FIG. 10 is a flowchart showing a procedure for monitoring a crossing area.
[Explanation of symbols]
1,1a Object recognition unit
2,2a Stereo camera
3,3a Main camera
4,4a Sub camera
5,5a A / D converter
6,6a A / D converter
7, 7a Image correction unit
8,8a Stereo image processing unit
9, 9a Image data memory
10, 10a Distance data memory
11, 11a microcomputer
12, 12a specific part
13, 13a setting part
14, 14a Object recognition unit
15 Monitoring unit

Claims (4)

In a railroad crossing monitoring device that monitors the situation inside railroad railroad crossings,
A stereo camera that captures a landscape including a monitoring area and outputs a pair of image data;
Distance data in which parallax is calculated by stereo matching based on the pair of image data, and a parallax group related to image data corresponding to one frame is associated with a coordinate position on an image plane defined by the image data Stereo image processing unit for outputting
In the image plane defined by the image data, a specifying unit for specifying the railroad rail,
A setting unit for setting a railroad crossing area for classifying an object projected in the monitoring area into a thing to be monitored and a thing not to be monitored based on the identified railroad rail,
The specifying unit extracts a pair of two-dimensional straight lines that are candidates for the rail in the image plane, and based on the distance data, calculates a three-dimensional straight line in real space corresponding to each of the two-dimensional straight lines. Calculating and determining whether or not the pair of three-dimensional straight lines are parallel, and when determining that the pair of three-dimensional straight lines are parallel, calculating an interval between the pair of three-dimensional straight lines, By comparing the calculated interval with a determination value corresponding to the actual interval between the railroad rails, it is determined whether the pair of two-dimensional straight lines that are candidates for the railroad rails are the railroad rails. A crossing monitoring device characterized by that. An object recognition unit for recognizing an object in the monitoring area based on the distance data;
Monitoring that determines whether or not the recognized object exists in the crossing area, and monitors the recognized object if it is determined that the object exists in the crossing area The crossing monitoring device according to claim 1, further comprising: a section. Correspondence between image data obtained by capturing a scene including a monitoring area with a stereo camera, a parallax group related to the image data equivalent to one frame output from a stereo image processing unit, and a coordinate position on an image plane defined by the image data based on the Tagged distance data, in railroad crossing monitoring method for monitoring the status of the crossing of the railroad rail,
In the image plane defined by the image data output from the stereo camera , the specifying unit extracts a pair of two-dimensional straight lines that are candidates for the railroad rails;
The specifying unit calculating a three-dimensional straight line in real space corresponding to each of the two-dimensional straight lines based on the distance data output from the stereo image processing unit ;
The identifying unit determining whether the pair of three-dimensional straight lines are parallel; and
When the specifying unit determines that the pair of three-dimensional straight lines are parallel, calculating a distance between the pair of three-dimensional straight lines;
The pair of two-dimensional straight lines that are candidates for the railroad rail are the railroad rails by comparing the calculated interval with a determination value corresponding to the actual interval of the railroad rail. Determining whether or not,
By the specifying unit, if the pair of two-dimensional straight line is determined to be the railway rail, the setting unit, based on the pair of two-dimensional linear, it monitors the target object projected on the monitoring area And a step of setting a level crossing area for classification into what should be monitored and what should not be monitored. An object recognition unit recognizing an object in the monitoring area based on the distance data;
A step of monitoring unit, the recognized object determines whether present in the crossing area,
The level crossing according to claim 3, further comprising a step of monitoring the recognized object when the monitoring unit determines that the target is present in the level crossing area. Monitoring method.

Images