JP6524529B2 - Building limit judging device - Google Patents

Description

  The present invention relates to an architectural limit determination device that determines a separation between an architectural limit area and an obstacle by image processing from a plurality of camera images in front of or behind a vehicle traveling on a rail in the railway field and the image processing field.

  As a device for detecting an obstacle by image processing of a camera image, in Patent Document 1, obstacle detection based on edge detection is performed. Moreover, in patent document 2, the area | region of a certain size obtained by the binarization process is detected as an obstruction.

Patent No. 4078798 Patent No. 4440809

Paul J. Besl, Neil D. McKay, "A Method for Registration of 3-D Shapes", IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. 14, NO. 2, FEBRUARY 1992, pp. 239-256 V. Kolmogorov, R. Zabih, "Multi-camera Scene Reconstruction via Graph Cuts", ECCV, VOL. 3, 2002, pp. 82-96

  However, the above-mentioned Patent Document 1 and Patent Document 2 have the following problems. Specifically, in Patent Document 1, only detection around the pantograph can be performed, and there is a problem that distance calculation by stereo measurement can not be performed for a portion where an edge can not be detected because of the edge-based method. Further, even in Patent Document 2, only detection around the pantograph can be performed, and processing such as binarization processing and mass detection is performed, so for objects that can not correspond to each processing, distance calculation by stereo measurement is There is a problem that it can not do.

  The present invention has been made in view of the above problems, and provides an architectural limit determination device that performs separation determination between an architectural limit area and an obstacle by performing distance calculation without using edge detection, binarization processing, and block detection. The purpose is to

An architectural limit judging device according to a first aspect of the present invention for solving the above-mentioned problems is:
A plurality of cameras for capturing, as image data, the front or the rear of a vehicle traveling on a rail;
A processing unit that performs arithmetic processing of a plurality of image data captured by the plurality of cameras;
The processing unit is
A distance calculation unit that calculates distance data of an obstacle by triangulation using block matching from a plurality of image data captured by the plurality of cameras and performs optimization of the distance data by graph cutting;
A rail recognition unit that calculates a rail position from a plurality of image data captured by the plurality of cameras;
A distance data integration unit that integrates the distance data of the plurality of points calculated by the distance calculation unit with respect to the image data captured at a plurality of points to obtain a three-dimensional environmental map and obtains a vehicle inclination;
Using the rail positions of the plurality of points calculated by the rail recognition unit, the vehicle inclinations of the plurality of points obtained by the distance data integration unit, and a static building limit area given as a fixed value A building limit area calculation unit for calculating a dynamic building limit area for the vehicle;
The separation distance from the building limit area to the nearest obstacle is calculated using the building limit area calculated by the building limit area calculation unit and the three-dimensional environmental map obtained by the distance data integration unit, and separation is performed. And a distance calculation unit that makes a determination.

An architectural limit judging device according to a second invention for solving the above-mentioned problems is:
In the building limit judging device according to the first invention,
The rail recognition unit performs raster scan matching on the line region of the image data in the rail direction of the rail using the template image data of the rail prepared in advance or the reference distribution data of the luminance value of the rail, A position to match is calculated as a rail position.

An architectural limit judging device according to a third invention for solving the above-mentioned problems is:
In the building limit judging device according to the first or second invention,
The distance data integration unit integrates the distance data of the plurality of points calculated by the distance calculation unit using an ICP (Iterative Closest Point) algorithm to obtain the three-dimensional environmental map and the vehicle inclination. It is characterized by asking.

  According to the present invention, since block matching and graph cutting are used, distance data can be calculated from monitored image data in consideration of smoothness regardless of edge detection, binarization processing, or block detection.

  Further, according to the present invention, since the rail position and the vehicle inclination are calculated, a dynamic building area for the vehicle is calculated using the rail position, the inclination, and the static building area given as a fixed value. It is possible to calculate In addition, since the three-dimensional environmental map and the building limit area are calculated, it is possible to calculate the separation distance of the obstacle and to determine the separation using the three-dimensional environment map and the building limit area.

  Further, according to the present invention, for a line region in the direction of an optional larch, matching is performed by raster scanning using template image data of rails or reference distribution data of luminance values of rails prepared in advance, and the most matching position Is calculated as the rail position, it is possible to detect the rail position accurately from the monitoring image data. In addition, distance data of surveillance image data at different shooting points are integrated by ICP algorithm to create a three-dimensional environment map, so it is possible to grasp the shape even for objects without design values, and additionally, ICP algorithm Calculation of vehicle inclination is also possible by

BRIEF DESCRIPTION OF THE DRAWINGS It is a perspective view which shows an example of the rail installation comprised by a rail and a utility pole. It is a figure which shows the area | region where distance data were able to be acquired in the imaging range from the train located in a certain point in the railway installation shown in FIG. It is a figure which shows the area | region where distance data were able to be acquired in the imaging range from the train located in a different point in the railway installation shown in FIG. It is a figure which shows the area | region which integrated the area | region of the acquired distance data shown in FIG.2 and FIG.3. It is a figure explaining calculation of separation isolation of a construction marginal field and an obstacle for which it asked from integrated distance data shown in Drawing 4. BRIEF DESCRIPTION OF THE DRAWINGS It is a figure which shows an example of embodiment of the building limit determination apparatus which concerns on this invention, (a) is a top view which shows the schematic structure, (b) is a figure which shows the imaged monitoring image. It is a block diagram explaining the process part which comprises the building limit determination apparatus shown to Fig.6 (a). It is a flowchart explaining the building limit judging method in the building limit judging device shown in Drawing 6 (a).

  Railways are large-scale transportation systems with large and huge facilities. Railway facilities require inspection and maintenance, and their automation is desired. As one of the automation, there is automatic determination by imaging with a camera and image processing. In the present invention, an architectural limit judging device is proposed as a new automatic measurement / maintenance function using a camera and image processing.

  In addition to the method of using a camera, there is also a method of using a laser or radar that can perform distance measurement with high accuracy for automatic measurement and maintenance, but there is a limit to the range of detection when the measured one can not be recognized or distinguished. May be On the other hand, in the case of an image, it is advantageous in terms of recognition and discrimination of measured things such as railway equipment and rails, and that the device is small and inexpensive compared to lasers and radars, resolution and imaging The camera according to the present invention, as shown in FIG. 6 to be described later, is superior in terms of speed, and it is possible for human to visually inspect the captured image because it has the possibility of development in the future. An apparatus configuration using the cameras 22a and 22b or the cameras 24a and 24b is adopted.

  As a method of determining the construction limit, a method using one camera can be considered, but in this case, only an object having a design value can be detected, and there is no design value or trees, etc. It was not possible to measure natural objects. In order to determine the architectural limit when there is no design value, it is effective to use a plurality of cameras instead of one and obtain the direct distance based on the principle of triangulation. In the present invention, it is effective to use FIG. As shown, the apparatus configuration is configured using a plurality of cameras 22a and 22b or cameras 24a and 24b.

  As a method of using a plurality of cameras, there are Patent Document 1 and Patent Document 2 described above, and the like. For example, in Patent Document 1 and Patent Document 2, in order to perform detection limited to the periphery of the pantograph, as described above, a method specialized for this (edge detection, binarization processing, and lump detection) is used, There is a restriction that distance calculation by stereo measurement can not be performed for an area where an edge can not be detected, and an object which can not cope with the binarization process and the block detection process. On the other hand, according to the present invention, by using processing to be described later different from Patent Document 1 and Patent Document 2, the entire area in front of the vehicle or the rear can be assumed without restriction described above. It is possible to target all objects such as natural objects without limitation.

  In order to detect an obstacle from an image of a camera, first, it is necessary to calculate the distance of the imaging range from a plurality of cameras. For example, as shown in FIG. 1, it is considered to acquire distance data of a railway installation composed of a rail 11 and a utility pole 12. Similar to FIG. 6 described later, when images are taken from a plurality of cameras 22a and 22b, as shown in FIG. 2, distance data is calculated using image data of the cameras 22a and 22b. When a plurality of cameras 22a and 22b exist and the relative position and orientation relationship between the respective cameras 22a and 22b is known, an obstacle (for example, an electric pole 12) is obtained from the cameras 22a and 22b according to the principle of triangulation. It is known that distance data up to) can be calculated.

  When determining the distance, it is necessary to find out which pixel on the image the pixel of interest on the image viewed by one camera is seen from another camera. As a method therefor, a method called block matching is often used in which an area similar to the peripheral pixels of a certain pixel of the image is searched from images of other cameras. This method works well when the region around the pixel of interest on the image has a characteristic texture, but when there is no texture, the block matching result is not uniquely determined, which causes the correct value not to be obtained. There is a case.

  In order to solve this problem, a method called a graph cut using a penalty term for optimization has been proposed so that the estimated distance of the pixel of interest does not differ greatly from the estimated distance of the surrounding (Non-Patent Document 2) . The method using block matching and graph cutting can perform distance calculation with high accuracy.

  Although the detection of an obstacle can be realized by using the above-described method, only with this, it is possible to obtain only distance data of one point taken. In order to determine the construction limit in the entire determination target section, it is necessary to create a three-dimensional map by integrating distance data taken from a plurality of points and to evaluate on the map.

  FIG. 3 shows an example of distance data acquisition when imaging is performed from a position different from that in FIG. In comparison with the results of FIG. 2, it can be seen that distance data of a part of the same section can be acquired. Here, in FIG. 2 and FIG. 3, the dotted line indicates the range in which the distance data can be acquired, and in FIG. 2, the distance data of the range A can be acquired and the distance data of the range B can be acquired in FIG. It is possible to integrate distance data by aligning two distance data using this common section. For alignment of distance data, a method called ICP (Iterative Closest Point) algorithm for aligning two three-dimensional point groups is often used (Non-Patent Document 1). Although the details will be described later, the present invention integrates the distance data by using the ICP, and finally creates a three-dimensional environmental map.

  For example, when range A of distance data in the case of imaging from a certain position shown in FIG. 2 and range B of distance data in the case of imaging from a different position from FIG. 2 shown in FIG. A three-dimensional environmental map can be created. In FIG. 4, a range C indicated by a dotted line indicates a range obtained by integrating the range A of distance data and the range B of distance data.

  Performing alignment of distance data is equivalent to determining relative position and orientation of distance data. By knowing the position and orientation, it is possible to obtain the amount of progress of the vehicle for each shooting, the speed, and the vehicle inclination. This vehicle inclination is a very important factor in defining the construction limit area. Incidentally, the ICP algorithm is a method of finding a solution by iterative calculation, and the result depends on the initial value. Therefore, although it is necessary to set the correct initial value, in the present invention, it becomes easy to set a good initial value by utilizing that the vehicle moves only on the rails.

Rail detection is as follows.
In order to determine the construction limit, it is necessary to recognize where the vehicle passes on the three-dimensional environmental map. It has been found that it is on the rails that the vehicle travels because the present invention has limited application to railways. Therefore, by recognizing the rails from the image, it is possible to obtain the passing area and the building limit area of the vehicle. In the present invention, raster scan is performed using the template image data of the rail prepared in advance or the reference distribution data (reference rail luminance distribution data) of the luminance value of the rail for the line area in the cross direction in the monitoring image of the front or rear. Matching is performed, and the most matching position is the rail position. Based on the rail position detected in this manner, calculation of the construction limit area is performed.

  The construction limit area is statically given as a fixed value, but changes dynamically when the vehicle tilts. Therefore, although the vehicle inclination is required for the calculation of the dynamic building limit area, this has already been obtained as the position and orientation of the vehicle by the ICP algorithm. Finally, the distance from the building limit area considering the vehicle inclination to the nearest obstacle at each point is calculated.

  For example, as shown in FIG. 5, the distance from the building limit area D in consideration of the vehicle inclination to the nearest obstacle at each point is calculated, and the building limit for the power pole 12 as the obstacle is calculated. The distance L at the point where the area D and the utility pole 12 come closest to each other is calculated.

  An architectural limit judging device according to the present invention in consideration of the above will be described with reference to FIGS.

Example 1
FIG. 6 is a view showing the building limit determination apparatus of the present embodiment, and FIG. 6 (a) is a top view showing its schematic configuration, and FIG. 6 (b) is a view showing a captured monitoring image . Moreover, FIG. 7 is a block diagram explaining the process part which comprises the building limit determination apparatus shown to Fig.6 (a). Moreover, FIG. 8 is a flowchart explaining the building limit determination method in the building limit determination apparatus shown to Fig.6 (a).

  As shown in FIG. 6A, the building limit determination apparatus of this embodiment includes two cameras 22a and 22b provided on a train 21 (vehicle) traveling on the rail 11 in the direction of arrow E, and a camera 22a. , 22b using the image M, and a processing unit 23 that performs separation determination between the building limit area and the obstacle by performing calculation processing described later.

  The cameras 22a and 22b are provided at the head of the train 21 and are disposed forward in the longitudinal direction of the rail 11 (in the direction of the arrow E). In this case, the front image M captured by the cameras 22a and 22b is It is used for the process mentioned later. Instead of the cameras 22a and 22b, cameras 24a and 24b provided at the end of the train 21 and disposed toward the rear in the longitudinal direction of the rail 11 may be used. In this case, the rear captured by the cameras 24a and 24b The image M of is used for the processing described later. The cameras 22a and 22b or the cameras 24a and 24b are fixed to the train 21. Depending on the movement, shaking, or tilting of the train 21, photographing of the surroundings is possible. The cameras 22a and 22b or the cameras 24a and 24b may be plural (two or more).

  The processing unit 23 described above, as shown in FIG. 7, includes a monitoring image input unit 31, a distance calculation unit 32, a rail recognition unit 33, a distance data integration unit 34, an architectural limit area calculation unit 35, A separation calculation unit 36 and a storage unit 37 are provided.

  The monitoring image input unit 31 inputs monitoring image data acquired by the plurality of front cameras 22a and 22b (or the plurality of rear cameras 24a and 24b) to the storage unit 37.

  The distance calculation unit 32 uses the monitoring image data to calculate the distance of the obstacle by a method using block matching and a graph cut, and inputs the calculated distance data to the memory unit 37.

  The rail recognition unit 33 uses the monitoring image data, performs matching using data prepared in advance described later, calculates the most matching position as the rail position, and inputs the rail position to the memory unit 37. ing.

  The distance data integration unit 34 uses the ICP algorithm to perform distance integration using all distance data calculated from all the captured monitoring image data groups, and three-dimensional environment map data and vehicle position and attitude data at each shooting position The three-dimensional environmental map data and the vehicle position and attitude data are input to the storage unit 37.

  The construction limit area calculation unit 35 calculates construction limit area data using the rail position, the vehicle position / posture data, and the static construction limit area given in advance, and the construction limit area data to the memory section 37 It is input.

  The separation calculation unit 36 calculates the separation between these data using the three-dimensional environmental map data and the construction limit area data, and inputs the separation data to the storage unit 37.

  The storage unit 37 stores the monitoring image data, distance data, three-dimensional environment map data, vehicle position and attitude data, rail position, building limit area data, and separation data described above.

  Next, with reference to FIG. 8, the procedure of the building limit determination method in the building limit determining apparatus having the above configuration will be described.

(Step S1)
The front (or rear) monitoring image data captured by the plurality of front cameras 22a and 22b (or the plurality of rear cameras 24a and 24b) is input to the storage unit 37 (monitoring image input unit 31). At this time, the monitoring image data may be input in association with the kilo information (or the kilo position) indicating the position of the train 21 on the rail 11.

(Step S2)
In the input monitoring image data, an obstacle is detected and a distance data of the obstacle is calculated by a method using triangulation using block matching and a graph cut (distance calculation unit 32).

  Specifically, using block matching, a region similar to a pixel around a certain pixel of interest of monitoring image data of one of the cameras 22a and 22b (or cameras 24a and 24b) is monitored by the other camera The data is searched, and the distance data is calculated for this target pixel using the principle of triangulation. If there is no characteristic texture around the pixel of interest on the monitoring image data, using a graph cut, the estimated distance data of the pixel of interest does not differ significantly from the estimated distance data of the surrounding, according to the penalty term. Optimize distance data.

(Step S3)
In the input monitoring image data, matching is performed using data to be described later prepared in advance, and rail recognition is performed with the most matching position as the rail position (rail recognition unit 33).

  Specifically, template image data of rails and reference rail luminance distribution data are prepared in advance for matching. Then, in the monitoring image data, matching by raster scan is performed using a template image data of rails and reference rail luminance distribution data for a line region in an arbitrary crosstie direction, and the most matching position is calculated as a rail position.

(Step S4)
If the imaging by the plurality of front cameras 22a and 22b (or the plurality of rear cameras 24a and 24b) is completed, the process proceeds to step S6. If the imaging is not completed, the process proceeds to step S5.

(Step S5)
If the imaging has not been completed, the front (or rear) surveillance image data newly imaged by the plurality of front cameras 22a and 22b (or the plurality of rear cameras 24a and 24b) is input ( Monitored image input unit 31), returns to step S2. That is, steps S2 to S5 are repeatedly performed until imaging is completed, and a monitoring image data group including a plurality of monitoring image data is input to the storage unit 37, and distance data and a rail position are obtained. Each monitoring image data of the monitoring image data group is an image captured from a different imaging point.

(Step S6)
All distance data of the acquired monitoring image data group are integrated using an ICP algorithm to obtain three-dimensional environmental map data, and vehicle position and attitude data (vehicle inclination). (Distance data integration unit 34).

  Specifically, with reference to monitoring image data captured at a certain shooting point, a plurality of points of the monitoring image data serving as a reference are extracted, three-dimensional coordinates of the plurality of points are set as initial values, and then different In the monitoring image data of the integration target captured at the shooting point, points corresponding to the plurality of points are determined, and using ICP algorithm, alignment between corresponding points is performed in two monitoring image data, 2 Integrate two surveillance image data. By performing this on all the acquired monitoring image data, all distance data of the monitoring image data group will be integrated, and three-dimensional environmental map data will be obtained.

  In addition, when distance data is integrated (alignment) using the ICP algorithm, the relative position and orientation of the distance data are determined. Vehicle position and attitude data (travel amount, speed, vehicle inclination) is also determined.

(Step S7)
The building limit area D is calculated based on the vehicle inclination obtained by the ICP algorithm in step S6 (see FIG. 5).

  Specifically, using the rail positions of the plurality of points calculated in step S3, the vehicle inclinations of the plurality of points obtained in step S7, and the static building limit area previously given as a fixed value, the train Calculate the dynamic building limit area D for.

(Step S8)
The separation distance L from the building limit area D at each point to the nearest obstacle (for example, the telephone pole 12) is calculated (see FIG. 5). Then, based on the calculated separation distance L, separation determination between the building limit area D and the obstacle is performed.

  As described above, since the building limit determination apparatus of this embodiment uses block matching and graph cutting, the smoothness is taken into consideration from the monitoring image data regardless of edge detection, binarization processing, and block detection. Calculation of distance data is possible.

  Further, as described above, the building limit determination apparatus of the present embodiment calculates the rail position and the vehicle inclination, so using the rail position, the vehicle inclination, and the static building limit area, the movement with respect to the train 21 is performed. It is possible to calculate typical architectural limit area data.

  Further, the building limit judging device of the present embodiment performs the matching by raster scan using the template image data of the rail or the reference rail luminance distribution data prepared in advance for the line region in the direction of the optional block direction, and performs the most matching. Since the position is calculated as the rail position, it is possible to detect the rail position accurately from the monitoring image data.

  In addition, since the building limit determination apparatus of this embodiment integrates distance data of monitoring images at different shooting points by using an ICP algorithm to create three-dimensional environmental map data, it is necessary to grasp the shape of an object having no design value. In addition, calculation of vehicle inclination is also possible by the ICP algorithm.

  Further, as described above, the building limit determination apparatus of the present embodiment calculates three-dimensional environment map data and building limit area data, so using the three-dimensional environment map data and building limit area data, an obstacle is generated. It is possible to calculate the separation distance and to determine the separation distance.

  The present invention is suitable for performing separation determination between an architectural limit area and an obstacle with respect to a vehicle traveling on a rail.

DESCRIPTION OF SYMBOLS 11 rail 12 power pole 21 train 22a, 22b, camera 23 process part 31 monitoring image input part 32 distance calculation part 33 rail recognition part 34 distance data integration part 35 construction limit area calculation part 36 separation calculation part 37 storage part

Claims (3)

A plurality of cameras for capturing, as image data, the front or the rear of a vehicle traveling on a rail;
A processing unit that performs arithmetic processing of a plurality of image data captured by the plurality of cameras;
The processing unit is
A distance calculation unit that calculates distance data of an obstacle by triangulation using block matching from a plurality of image data captured by the plurality of cameras and performs optimization of the distance data by graph cutting;
A rail recognition unit that calculates a rail position from a plurality of image data captured by the plurality of cameras;
A distance data integration unit that integrates the distance data of the plurality of points calculated by the distance calculation unit with respect to the image data captured at a plurality of points to obtain a three-dimensional environmental map and obtains a vehicle inclination;
Using the rail positions of the plurality of points calculated by the rail recognition unit, the vehicle inclinations of the plurality of points obtained by the distance data integration unit, and a static building limit area given as a fixed value A building limit area calculation unit for calculating a dynamic building limit area for the vehicle;
The separation distance from the building limit area to the nearest obstacle is calculated using the building limit area calculated by the building limit area calculation unit and the three-dimensional environmental map obtained by the distance data integration unit, and separation is performed. An architectural limit judging device characterized by having a distance calculation part which judges. In the building limit judging device according to claim 1,
The rail recognition unit performs raster scan matching on the line region of the image data in the rail direction of the rail using the template image data of the rail prepared in advance or the reference distribution data of the luminance value of the rail, An architectural limit judging device characterized by calculating a matching position as a rail position. In the building limit judging device according to claim 1 or 2,
The distance data integration unit integrates the distance data of the plurality of points calculated by the distance calculation unit using an ICP (Iterative Closest Point) algorithm to obtain the three-dimensional environmental map and the vehicle inclination. An architectural limit judging device characterized by asking.

Images