JP2002002485A - Track recognition device and railway rolling stock using track recognition device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To precisely recognize a track without being affected by the external light, etc. SOLUTION: A rail candidate point extraction part 40 evaluates the sequence of luminance in each pixel in a plurality of directions to extract a rail candidate point and finds the sequence direction of the luminance of the rail candidate point. A reference point detection part 51 detects a rail reference point to be the reference point when detecting an own rolling stock track, by directly detecting the rail candidate point. A rail tracking part 52 follows a rail typical point extending to the rail reference point. In this case, a rail typical point on a search line is estimated for every search line set on an image based on the previously decided rail typical point in this side and a final rail typical point is decided out of a prescribed range on the search line with the estimation position set to the center, based on a prescribed evaluation index.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a track recognizing apparatus for recognizing a track of an own vehicle using an image of a traveling direction captured by an image pickup means mounted on a vehicle running on a track, and a railway using the track recognizing apparatus. For vehicles.

[0002]

2. Description of the Related Art Generally, in a railway vehicle, when a maintenance vehicle or the like travels on a business main line after the end of commercial operation, a "train central control (ATC / ATS)" device used for business is not used. It is necessary for the passenger to proceed while checking the path of the vehicle.

As a technique for reducing the burden on the passenger and improving safety, for example, Japanese Patent Application Laid-Open No. 2000-032601 by the present applicant discloses a technique based on data obtained by a CCD camera or the like. Generate an image of the vehicle traveling direction,
For each scanning line of the image, extract a region that is in a predetermined state with respect to the average value of the surrounding luminance and set rail candidate points,
A technology for recognizing a trajectory of a vehicle traveling based on set rail candidate points has been disclosed.In addition, such a trajectory recognition device is applied to a railway vehicle such as a maintenance vehicle to control a vehicle or to control a passenger. For performing display control of trajectory information with respect to a vehicle.

[0004]

However, as disclosed in the above-mentioned Japanese Patent Application Laid-Open No. 2000-032601, when a rail candidate point is extracted by a so-called absolute value evaluation of luminance, the influence on the rail due to external light or the like is increased. If the absolute luminance changes, it is difficult to sort the rails and the laying surface (extraction of rail candidate points), and the accuracy of track recognition may be reduced.

SUMMARY OF THE INVENTION The present invention has been made in view of the above circumstances, and provides a track recognizing device capable of accurately performing track recognizing without being affected by external light or the like, and a railroad vehicle using the track recognizing device. With the goal.

[0006]

According to a first aspect of the present invention, there is provided a trajectory recognizing device which continuously scans a vehicle along a trajectory by using an image of a traveling direction of a vehicle captured by an image pickup means. A track recognition device for detecting a fixed object disposed in the vehicle and recognizing the own vehicle trajectory, wherein the fixed point extracts a fixed object candidate point by evaluating the continuity of luminance at each pixel on the image. It is characterized by comprising object candidate point extracting means, and fixed object detecting means for detecting a fixed object arranged on the own vehicle track based on the image and the fixed object candidate points.

According to a second aspect of the present invention, in the trajectory recognizing apparatus according to the first aspect, the fixed object candidate point extracting means includes means for extracting luminance in a plurality of directions with respect to a pixel of interest. The continuity is evaluated, and it is determined whether or not the pixel of interest is a fixed object candidate point based on the evaluation value of the continuity of luminance in a plurality of directions.

According to a third aspect of the present invention, in the trajectory recognition apparatus according to the first or second aspect, the fixed object candidate point extracting means includes a plurality of fixed object candidate point extracting means different for a pixel of interest. A small area of pixels connected in the search direction is set, and a difference value of luminance in each of the small areas is used as an evaluation value of the continuity of luminance of the target pixel in each search direction.

According to a fourth aspect of the present invention, in the trajectory recognition apparatus according to the first or second aspect, the fixed object candidate point extracting means includes a plurality of different fixed object point extracting means for a pixel of interest. A small area of pixels connected in the search direction is set, and a variance value of the luminance in each of the small areas is set as an evaluation value of the continuity of the luminance of the target pixel in each search direction.

According to a fifth aspect of the present invention, in the trajectory recognition apparatus according to the first or second aspect, the fixed object candidate point extracting means comprises a plurality of different fixed object candidate point extracting means for a target pixel. A small area of pixels connected in the search direction is set, and the larger of the difference value and the variance value of the luminance in each of the small areas is used as the evaluation value of the continuity of the luminance of the target pixel in each search direction. It is characterized by.

According to a sixth aspect of the present invention, in the trajectory recognizing device according to the third aspect of the present invention, the length of the small area in the search direction is set in the search direction on the image. The width is set to be longer than the width of the fixed object.

In the trajectory recognition apparatus according to the present invention, the fixed object candidate point extracting means may include a plurality of directions of the target pixel. The minimum value and the maximum value are determined from among the evaluation values of the continuity of the luminance, and based on the minimum value and the maximum value of the evaluation value of the continuity of the luminance, whether the target pixel is a fixed object candidate point It is characterized in that it is determined whether or not it is.

In the trajectory recognition apparatus according to the present invention, the fixed object candidate point detecting means may include a minimum value of the evaluation value of the brightness continuity, It is characterized in that it is determined whether or not the pixel of interest is a fixed object candidate point based on the difference between the maximum value and the minimum value of the evaluation value of the continuity of luminance.

The trajectory recognition device according to the ninth aspect of the present invention is the trajectory recognition device according to any one of the second to eighth aspects, wherein the fixed object candidate point extracting means is configured to determine a luminance of the target pixel. The continuous direction of the luminance at the extracted fixed object candidate point is obtained based on the evaluation value of the continuity.

According to a tenth aspect of the present invention, in the trajectory recognition device according to any one of the first to ninth aspects, the three-dimensional object detecting means for detecting the three-dimensional object in the traveling direction of the own vehicle is provided. The fixed object detecting means detects a three-dimensional object in a predetermined height range based on the three-dimensional object information detected by the three-dimensional object detecting means, and the pixel corresponding to the three-dimensional object in the predetermined height range And removing the fixed object candidate points from the fixed object detection processing target.

In the trajectory recognition apparatus according to the eleventh aspect, in the invention according to any one of the first to tenth aspects, the fixed object detecting means is based on the fixed object candidate points. Reference point detection means for detecting a fixed object reference point which is a reference at the time of fixed object detection, and fixed object tracking means for sequentially detecting fixed object representative points connected to the fixed object reference point, It is characterized by having.

According to a twelfth aspect of the present invention, in the trajectory recognition apparatus according to the eleventh aspect, the reference point detecting means sets a reference point search area in front of the own vehicle, and The fixed object reference point is detected based on the fixed object candidate points distributed in the search area.

In the trajectory recognition apparatus according to a thirteenth aspect of the present invention, in the twelfth aspect of the present invention, the reference point detecting means detects the fixed object candidate points distributed in the reference point search area. On the other hand, a straight line detection process is performed for each of the fixed object candidate points having the same luminance continuous direction, and the fixed object reference point is detected based on the detected straight line.

In the trajectory recognition apparatus according to the present invention, the fixed object tracking means may be arranged on the image at predetermined intervals. A search line is set, and the fixed object search points within a predetermined range set on this search line are evaluated based on the fixed object reference point or the fixed object representative point detected in the foreground, and the next fixed object is evaluated. It is characterized by detecting a representative point.

According to a twelfth aspect of the present invention, in the trajectory recognizing device according to the fourteenth aspect, the distance between the search lines is set to be narrower as the distance from the host vehicle increases. It is characterized by.

The trajectory recognition device according to the invention of claim 16 is the invention according to claim 14 or claim 15, wherein the fixed object tracking means includes a fixed object reference point or a fixed object detected before. The predicted position of the fixed object representative point on the next search line is calculated based on the object representative point, and the degree of coincidence of each fixed object search point on the search line with the predicted position is used as an evaluation index. Features.

In a trajectory recognition apparatus according to a seventeenth aspect of the present invention, in the invention according to any one of the fourteenth to sixteenth aspects, the fixed object tracking means includes a fixed object representative detected in front. It is characterized in that the degree of coincidence between the luminance pattern of the set area centered on the point and the luminance pattern of the set area centered on each fixed object search point on the next search line is used as an evaluation index.

The trajectory recognition device according to the invention of claim 18 is the invention according to any one of claims 14 to 17, wherein the fixed object tracking means includes a fixed object representative detected in front. It is characterized in that the degree of coincidence between the direction of each fixed object search point on the next search line with respect to the point and the continuous direction of luminance at each fixed object search point is used as an evaluation index.

According to a twelfth aspect of the present invention, in the trajectory recognition device according to the eleventh aspect, the fixed object is a pair of right and left rails, and the reference inspection is performed. The output means detects a pair of rail reference points as the fixed object reference point, and the fixed object tracking means sequentially detects a rail representative point connected to the pair of rail reference points as the fixed object representative point. Features.

In the trajectory recognizing device according to the twentieth aspect, the reference point detecting means may be based on a theoretical inter-rail distance obtained from an actual inter-rail distance. It is characterized in that the pair of rail reference points are detected.

According to a twelfth aspect of the present invention, in the trajectory recognition apparatus according to the nineteenth or twentieth aspect, the fixed object tracking means complements the left and right rail representative points with each other. Features.

Further, the maintenance vehicle using the track recognition device according to the invention described in claim 22 is the maintenance vehicle according to claims 1 to 21.
A railroad vehicle using the track recognition device according to any one of claims 1 to 4, wherein the control performs at least one of vehicle control and display control of track information to a passenger based on the track information recognized by the track recognition device. Means are provided.

[0028]

Embodiments of the present invention will be described below with reference to the drawings. The drawings relate to an embodiment of the present invention. FIG. 1 is a functional block diagram showing a track recognition device mounted on a tracked vehicle, FIG. 2 is a flowchart of rail candidate point extraction processing, and FIG. Illustrated illustration, FIG.
Is an explanatory diagram showing a small area when evaluating the continuity of luminance in each search direction, FIG. 5 is a chart showing a relationship between a search direction of luminance continuity and its evaluation value, and FIG. FIG. 7 is a flowchart of a reference point detection process, FIG. 8 is an explanatory diagram showing a reference point search area in the direction image, FIGS. 9 and 10 are flowcharts of a rail tracking process, and FIG. FIG. 4 is an explanatory diagram showing a search line.

In FIG. 1, reference numeral 1 denotes a vehicle traveling on a track such as a railway, and in this embodiment, a vehicle for maintenance and inspection around the track. The vehicle 1 has a track (hereinafter, referred to as a track) on which the vehicle travels by detecting a vehicle traveling rail as a fixed object.
A trajectory recognition device 2 for recognizing the own vehicle trajectory is mounted. The trajectory recognition device 2 includes a stereo camera 10 as an imaging unit that captures an object in the traveling direction of the vehicle 1 from different viewpoints, and an image that generates a pair of images (also referred to as an original image) based on signals from the stereo camera 10. Intake unit 20
And a distance image generation unit 30 that performs stereo processing on a pair of original images to generate a distance distribution (distance image) over the entire image.
And a three-dimensional object detection unit 35 that detects a three-dimensional object based on a distance image, and a fixed object that evaluates the continuity of luminance at each pixel of one image to extract rail candidate points as fixed object candidate points Rail candidate point extraction unit 40 as candidate point extraction means
And an own-vehicle running rail detection unit 50 as a fixed object detecting means for detecting the own-vehicle running rail as a fixed object based on the rail candidate points.

The own-vehicle-traveling-rail detecting section 50 serves as reference point detecting means for detecting a pair of rail reference points as fixed object reference points based on rail candidate points. A reference point detecting unit 51 and a rail tracking unit 52 as fixed object tracking means for sequentially detecting rail representative points as fixed object representative points sequentially connected to the rail reference point and recognizing the own vehicle trajectory. It is configured.

Here, the information about the own vehicle track recognized by the rail tracking section 52 is output to the control section 100 as control means of the vehicle 1, and the control section 100 performs various vehicle control and boarding based on the track information. Orbit information is displayed to the user.

Two CCs constituting the stereo camera 10
The D cameras 10a and 10b are monochrome or color CCD cameras synchronized with each other.
The camera 10a is a main camera that captures a reference image during stereo processing, and the other CCD camera 10b is a sub-camera that captures a comparison image during stereo processing. It is arranged to become.

The image capturing section 20 includes a CCD camera 10a,
The two-system analog signals from 10b are converted into digital image data of a predetermined luminance gradation to generate a pair of images (original images). In this case, it is assumed that the upper, lower, left and right of the image are equal to the upper, lower, left and right of the image, and the image coordinate system is such that the upper left of the image is the origin, the horizontal scanning direction is the X axis, and the vertical scanning direction is the Y axis.

The distance image generation section 30 specifies a small area corresponding to the original image by calculating a city block distance for each small area of the reference image and the comparison image and obtaining a correlation between them. Stereo matching is performed, and a distance image is generated using distance (= parallax) of pixels generated according to the distance to the object as distance data.

The three-dimensional object detection unit 35 uses the distance image
A three-dimensional object is detected by generating a histogram in the distance direction for the strip-shaped region. The above-described generation of a distance image and detection of a three-dimensional object are described in, for example, Japanese Patent Application Laid-Open No. Hei 6-266 by the present applicant.
No. 828 discloses this in detail.

Next, the rail candidate point extraction processing by the rail candidate point extraction unit 40 will be described with reference to the flowchart shown in FIG. Here, the rail candidate point extracting unit 40
Is inputted with one of a pair of images (reference image) stored in an image memory in the image capturing unit 20, and sequentially enters each pixel within a designated image range (see FIG. 3) on the image. Attention is paid to determine whether or not the pixel of interest is a rail candidate point.

In the rail candidate point extracting process, first, in step S101, a certain pixel among the pixels within the designated image range on the image is focused on, and a predetermined number of pixels continuous in the left-right direction with the current focused pixel as a center. It is checked whether or not the amount of change in the luminance of the pixel is equal to or greater than a set value.

Here, when the traveling direction of the vehicle 1 is imaged, the own vehicle traveling rail always extends substantially upward starting from the lower part of the image due to the characteristics of the rail. Further, since the curvature when the rail curves is generally small, it is unlikely that the own vehicle running rail continues in the left and right direction of the image even when the own vehicle running rail is curved. Further, a rail that merges or branches with the own vehicle travel rail gently merges or branches with the own vehicle travel rail. Therefore, it is unlikely that such a rail continues in the left-right direction of the image. Therefore, it is difficult to imagine that the rails related to the own vehicle trajectory are continuous in the left and right direction of the image. When the pixel of interest represents the rail related to the own vehicle trajectory, a pixel row that is continuous in the left and right direction of the pixel of interest crosses the rail. . In such a case, since the brightness of the imaged rail surface is clearly different from the brightness of the laying surface or the like, the change in the brightness of each pixel in the pixel row becomes large.

In step S101, if it is determined that the amount of change in the luminance of the pixel row continuing in the left-right direction of the pixel of interest is smaller than the set value, the pixel of interest this time represents a rail related to the own vehicle trajectory. It is determined that the pixel is not a pixel, the process proceeds to step S102, and the current pixel of interest is excluded from the rail candidate points, and then the process proceeds to step S108.

On the other hand, if it is determined in step S101 that the amount of change in the luminance of the pixel row that is continuous in the left-right direction around the pixel of interest is equal to or greater than the predetermined value,
Go to 103. In step S103, the continuity of luminance in a plurality of directions for the pixel of interest is evaluated. in this case,
For example, eight directions (−90 ° (90
°), -68 °, -45 °, -23 °, 0 °, 23 °,
(45 °, 68 °).

More specifically, in step S103, first, as shown in FIGS. 4A to 4H, for example, 3 pixels continuous in each search direction centering on the pixel of interest (indicated by a bold frame in the figure) A small area of × 5 pixels is set. The length of the small area in the longitudinal direction is set to be larger than the rail width shown on the image.

Then, for each small area, the average value (difference value Diff) of the difference in luminance between the adjacent pixels and the variance value De of the luminance are calculated.
v is calculated, and the larger one of them is set as the evaluation value of the continuity of the luminance in the search direction of the pixel of interest.

Here, assuming that the brightness of each pixel in the small area is Pm, n (m = 0 to 5, n = 0 to 2) as shown in FIG. 4, the difference value Diff is, for example,

(Equation 1) And the variance Dev is, for example,

(Equation 2) However,

(Equation 3) It is calculated by In addition, as is clear from each equation,
The evaluation value becomes smaller as the luminance continuity becomes higher.

Next, in step S104, the maximum value and the minimum value are extracted from the evaluation values of the continuity of the luminance in each search direction for the target pixel.

In the following steps S105 and S106, it is determined whether or not the target pixel is a rail candidate point based on the extracted maximum value and minimum value of the evaluation value.

Here, when the pixel of interest represents a rail, the continuity of luminance is increased only in the direction in which the rails are continuous, and the continuity in other directions across the rail is reduced. Therefore, when the pixel of interest is a rail candidate point, FIG.
As shown in (1), the evaluation value of the luminance continuity shows a low value only in a certain search direction, and the evaluation value in another search direction shows a high value.

When the target pixel represents a laid surface such as gravel or the like, the continuity of luminance in all directions is low. Therefore, as shown in FIG. The evaluation value shows a high value.

When the pixel of interest represents a laying surface such as concrete, the luminance continuity is high in all directions. Therefore, as shown in FIG. The evaluation value shows a low value.

Therefore, first, in step S105, it is checked whether or not the minimum value of the evaluation value is larger than the specified value a1 to check whether or not the luminance of the target pixel has predetermined continuity. Is determined to be larger than the specified value a1, the target pixel is determined to be a pixel indicating a gravel or the like having low continuity of luminance in all directions, and the process proceeds to step S102.

On the other hand, if it is determined in step S105 that the minimum value of the evaluation value is equal to or smaller than the specified value a1, the process proceeds to step S106, and the difference between the maximum value and the minimum value of the evaluation value is smaller than the specified value a2. Check whether or not. That is,
In step S106, it is checked whether or not the luminance continuity is specialized in a predetermined direction in the target pixel having the luminance continuity.

If it is determined in step S106 that the difference between the maximum value and the minimum value of the evaluation value is smaller than the specified value a2, the target pixel is a concrete or the like having a high luminance continuity in all directions. And the process proceeds to step S102.

On the other hand, if it is determined in step S106 that the difference between the maximum value and the minimum value of the evaluation value is equal to or larger than the specified value a2, the pixel of interest has luminance continuity specialized in a certain direction. It is determined that it is a rail candidate point, and the process proceeds to step S107.

In the following step S107, a continuous direction of luminance at the target pixel determined to be a rail candidate point is detected. This continuous direction detection of the luminance is performed based on the evaluation value of the luminance continuity in a plurality of directions of the pixel of interest, which is a rail candidate point, and the direction in which the evaluation value is the minimum is set as the luminance continuous direction. You.

When the process proceeds from step S102 or S107 to step S108, step S1 is executed.
At 08, it is checked whether or not the next pixel of interest is within the specified image range, that is, whether or not there is a pixel that has not been determined as a rail candidate point within the specified image range. If it is determined that the target pixel is the next pixel, the process returns to step S101. On the other hand, if it is determined that there is no undetermined pixel in the designated image range, the routine exits from the routine.

By such processing, the rail candidate point extracting unit 40, as shown in FIG. 6, has rail candidate point distribution information (like an image) composed of rail candidate points having information on the luminance continuous direction. Direction image).

Next, the reference point detection processing of the own vehicle running rail by the reference point detection section 51 will be described with reference to the flowchart shown in FIG. The reference point detector 51 receives information about the three-dimensional object from the three-dimensional object detector 35 and information about the directional image from the rail candidate point extractor 40, and sets the information below the directional image (at the foot of the vehicle 1). The coordinates of the left and right vehicle running rails on the reference point search line thus detected are detected, and the detected pair of coordinates is used as a rail representative point (hereinafter referred to as a rail reference point) as a reference when tracking the vehicle running rail. ). Here, the detection of the rail reference point is performed based on the rail candidate points in the reference point search area as shown in FIG. 8, and the reference point search area is set along the reference point search line. It is.

When the reference point detection process starts, first,
In step S201, a rail candidate point corresponding to a three-dimensional object having a predetermined height (predetermined height) is removed from the rail candidate points in the reference point search area. That is, a three-dimensional object having a predetermined height with respect to the laying surface has a high possibility of a wall surface, a pillar, a traffic light, a side groove, and the like, and has a low possibility of being a rail. Therefore,
In step S201, first, a three-dimensional object having a predetermined height is extracted from the three-dimensional objects detected by the three-dimensional object detection unit 35,
By excluding the rail candidate points corresponding to the extracted three-dimensional object from the target of the reference point detection processing, the processing speed is increased and the detection accuracy is improved.

In the following step S202, it is checked whether or not the current reference point detection process is a process immediately after the system startup. If it is determined that the current process is a process immediately after the system startup, the process proceeds to step S205.

On the other hand, if it is determined in step S202 that the current process is not the process immediately after the system startup, the process proceeds to step S203, and it is determined whether or not the rail reference point has been detected in the previous process. If it is determined that the rail reference point has been detected in the previous processing, the process proceeds from step S203 to step S204. If it is determined that the rail reference point has not been detected in the previous processing, the processing proceeds from step S203 to step S203. Proceed to S205.

In step S204, the previously detected rail reference point is set as the predicted point of the current rail reference point detection, and the flow advances to step S206.

On the other hand, in step S205, the theoretical rail coordinates on the reference point search line when the vehicle 1 is on the straight rail are set as the predicted points of the current rail reference point detection, and then the process proceeds to step S206. That is, if the current process is a process immediately after system startup or if no rail reference point is detected in the previous process, the vehicle 1
Is assumed to be on a straight rail, and the rail coordinates on the reference point search line uniquely determined from the mounting position and mounting direction of the camera are substituted as the predicted points for the current rail reference point detection.

In step S206, HOUGH conversion processing (straight line detection processing) is performed on rail candidate points that have the same luminance continuous direction and are linearly distributed in the reference point search area.

In the following step S207, step S2
Based on the data of the straight line group detected in step 06, a coordinate pair as a candidate for the left and right rail reference points is obtained. Specifically, focusing on the fact that the distance between the left and right rails is constant and the left and right rails are parallel to each other in the real space, step S2
From the straight line group detected at step 06, a pair of straight lines whose distance from each other theoretically corresponds to the actual rail-to-rail distance on the image and which are parallel to each other are selected, and the selected straight line pair intersects the reference point search line. The coordinate pair is set as a candidate for the left and right rail reference points.

In the following step S208, from the coordinate pairs of the extracted rail reference point candidates, the one closest to the prediction point set in step S204 or S205 is set as the current rail reference point, and the routine exits. .

Next, the rail tracking processing by the rail tracking section 52 will be described with reference to the flowchart shown in FIG. The rail tracking unit 51 extracts one of a pair of images (reference image) stored in an image memory in the image capturing unit 20, information on a three-dimensional object from the three-dimensional object detection unit 35, and extraction of rail candidate points. Information on the direction image from the unit 40 and information on the rail reference point from the reference point detection unit 51 are input, and based on the reference image and the direction image, the representative points of the own vehicle traveling rail are sequentially set in the distance from the rail reference point to the starting point. A rail tracking process for performing a search is performed.

Specifically, for each search line (see FIG. 11) set on the image, a rail representative point on the search line is predicted based on the previously determined rail representative point on the near side. A final rail representative point is determined from a predetermined range on the search line centered on the predicted position based on three evaluation indices described later. Here, in the present embodiment, the intervals between the search lines are set to be narrower as the distance from the vehicle 1 increases.

When the rail tracking process starts, first,
In step S301, in order to increase the speed and accuracy of the rail tracking process, pixels corresponding to a three-dimensional object having a predetermined height (a predetermined height) are removed from the pixels on the image, and a rail candidate point extraction unit is used. A rail corresponding to a three-dimensional object having a predetermined height (predetermined height) is removed from the rail candidate points on the direction image extracted at 40.

In the following step S302, it is checked whether or not the search line for the current rail representative point search is a search line located near the vehicle 1, and it is determined that the current search line is near the vehicle 1. In this case, step S303
Proceed to.

In step S303, a rail representative point (coordinates) on the current search line is predicted based on the previously determined rail representative point immediately before. This predicted position is determined by detecting the intersection of a straight line extending in the continuous direction of the luminance of the rail representative point from the preceding rail representative point and the current search line.

On the other hand, if it is determined in step S302 that the current search line is a search line located a predetermined distance from the vehicle 1, the process proceeds to step S304. In step S304, a rail representative point on the current search line is predicted based on a few rail representative points before. The prediction of the rail representative point is performed by linearly predicting the rail position on the current search line from the rail representative points in the foreground.

Following steps S305 to S309
The processing up to step S303 or step S30
A final rail representative point is evaluated and determined based on the evaluation index from a pair of left and right search areas respectively set on the search line with the predicted position of the left and right rail representative points predicted in 4 as a center. is there. As the evaluation index, an evaluation value of the continuity of the position from the near side (step S305),
Evaluation value of the continuity of the luminance pattern from the front (step S
306) and the evaluation value of the continuity of the directivity from the near side (step S307).

In step S 305, first, for each rail candidate point (hereinafter referred to as a rail search point) existing in the search area, an evaluation indicating the continuity of the position from the previously determined rail representative point before the rail. As the value E0 (x), the degree of coincidence (deviation) of the predicted position with respect to the rail representative point is obtained.

Here, the evaluation value E at each rail search point
0 (x) is calculated by the following equation, for example.

[0074]

(Equation 4) Here, X: predicted position of the rail representative point on the search line x: position of each rail search point on the search line That is, the evaluation value E0 (x) is such that each rail search point is shifted from the predicted position of the rail representative point. The larger the value.

In the following step S306, as an evaluation value E1 (x) indicating the continuity of the luminance pattern from the front, the luminance pattern in the set area centered on the previously determined rail representative point is determined. The matching degree (deviation) of the luminance pattern in the set area centered on each rail search point on the current search line is obtained.

That is, focusing on the fact that the luminance pattern around the rail is similar at any position in the longitudinal direction of the rail, the luminance pattern in the set area centered on the front representative point and the respective rail search points are determined. By evaluating the degree of coincidence with the luminance pattern in the center set area, it is attempted to evaluate the possibility that each rail search point is a rail representative point. Here, in the present embodiment, in consideration of the rail width on the image,
For example, the region is set to be a 5 × 3 pixel region that is slender on both sides.

Evaluation value E1 (x) at each rail search point
Is calculated by the following equation, for example.

[0078]

(Equation 5) Here, xr, yr: the coordinates of the rail representative point on the previous search line x, y: the coordinates of each rail search point Px, y: the luminance at each rail search point In the next step S307, the continuity of the direction from the near side Is determined by comparing the direction of each rail search point with respect to the preceding rail representative point and the continuous direction of luminance at the preceding rail representative point as the evaluation value E2 (x).

Here, the evaluation value E at each rail search point
2 (x) is calculated by the following equation, for example.

[0080]

(Equation 6) In the following step S308, comprehensive evaluation is performed based on the evaluation values E0 (x), E1 (x), E2 (x) for each rail search point, and a final rail representative point is determined. That is, the evaluation values E0 (x), E1 (x),
An evaluation value E (x) obtained by predetermined summation of E2 (x) is obtained, and a rail search point having the smallest evaluation value E (x) is set as a final rail representative point (coordinate). However, when the evaluation value E (x) does not satisfy the condition of the predetermined threshold value, it is assumed that the rail representative point has not been detected.

Then, in step S309, it is checked whether or not there is a pixel whose luminance is lower than the pixel representing the rail representative point in an area within one pixel on the left and right of the rail representative point obtained in step S308, and the rail representative point is represented. If it is determined that there is a pixel whose luminance is lower than that of the pixel, the coordinates of the rail representative point are corrected to the coordinates of the pixel having the lowest luminance, and the process proceeds to step S310.

The correction in step S309 is caused, for example, by the fact that when the rail is imaged at night, the mirror surface of the rail is generally darkened on the laying surface or the like. The mirror surface of the rail may appear brighter than the installation surface. Therefore, under the imaging conditions in which the mirror portion of the rail is brightly displayed on the laying surface or the like, the coordinates of the rail representative point obtained in step S308 are changed to the coordinates of the brightest pixel in the area within one pixel on either side. It may be corrected.

In step S310, step S308
In, it is checked whether only one of the left and right rail representative points has been detected. If it is determined that only one of the rail representative points has been detected, the process proceeds to step S311.

In step S311, from one of the left and right rail representative points that has been detected, the other rail representative point is complementarily set using the left and right rail intervals and the relative difference in the rail direction. Proceed to S312.

In the following step S312, it is checked whether or not the state in which the rail representative point on the same side cannot be detected has continued for a predetermined number of times. If it is determined that the error has occurred, the process exits the routine. That is, if the rail representative point on the same side cannot be detected continuously for the specified number of times, the reliability of the subsequently detected rail representative point decreases, and the rail tracking processing ends.

On the other hand, if it is determined in step S310 that the detection of the rail representative point is not the detection of only one, the process proceeds to step S313. In step S313, both the left and right rail representative points cannot be detected in step S308. Check if there was.

If it is determined in step S313 that both the left and right rail representative points cannot be detected, the flow advances to step S314 to proceed to step S30.
3 or after performing the rail representative point complementing process using the predicted position of the rail representative point obtained in step S304, the process proceeds to step S315.

In the following step S315, it is checked whether or not the state in which the left and right rail representative points cannot be detected has continued for a specified number of times. If it is determined that the specified number has not been continued, the process proceeds to step S318, while If it is determined that the error has occurred, the process exits the routine. That is, if the left and right rail representative points cannot be detected continuously for the specified number of times, the reliability of the subsequently detected rail representative points decreases, and the rail tracking processing ends.

On the other hand, in step S313, it is determined that both the left and right rail representative points have been detected, and the process proceeds to step S31.
In step S316, it is determined whether or not the detected distance between the left and right rail representative points is an appropriate value for the rail distance on the image based on the rail distance in the real space. When it is determined that the interval between the representative points is an appropriate value, the process proceeds to step S318, and when it is determined that the interval between the rail representative points is not an appropriate value, the process proceeds to step S317.

In step S317, the rail representative point having the larger evaluation value E (x) among the left and right rail representative points is invalidated, and the process advances to step S311 to proceed to the rail representative point which has not been invalidated (one of the rail representative points). From the representative point), the other rail representative point is complemented by using the left and right rail intervals and the relative difference in the rail direction.

When the process proceeds from step S312, step S314, or step S315 to step S318,
In step S318, it is checked whether or not the search processing of the rail representative point has been performed up to the specified search line located far from the own vehicle. If it is determined that the search of the rail representative point has been performed up to the specified search line, , And exit the routine.

On the other hand, if it is determined in step S318 that the search processing of the rail representative point has not been performed up to the specified search line, the process returns to step S302, and the same processing is performed for a search line one distance farther from the current search line. Repeat the search process.

The search for a rail representative point in the present embodiment described above is performed using a rail candidate point existing in a search area centered on the predicted position of the rail representative point as a search point. Pixel positions that exist in the search area but are not set as rail candidate points may also be set as rail search points and evaluated. In this case, since the pixel position is not a rail candidate point, the pixel position does not have data on the continuous direction of luminance, and the evaluation value E2 (x) cannot be calculated. Therefore, the evaluation value E2 (x) at that pixel position
Is a fixed value set in advance.

According to such an embodiment, the continuity of luminance on an image is evaluated for each pixel, and rail candidate points are extracted based on the evaluation value of the continuity of luminance.
Separation of the laying surface and the like from the rail candidate points can be performed with high accuracy.

That is, a pixel representing a rail is in a continuous direction of the rail with respect to the periphery (a direction in which the mirror surface of the rail is continuous).
Focusing on the fact that the continuity of the brightness is high only for the pixels, and by extracting the rail candidate points by evaluating the continuity of the brightness in multiple directions for each pixel, the absolute brightness of the rails may be partially Even in the case of a change, for example, the extraction result of the rail candidate points is hardly affected by external light or the like.

Further, the evaluation of the continuity of the luminance in the target pixel is performed by setting a small area of pixels connected to a plurality of different search directions centering on the target pixel, and setting at least one of the luminance difference value and the variance value in this small area. Since it is performed by calculating either one of them, it can be performed easily and in a short time as compared with the evaluation of the continuity of luminance using frequency transformation such as FFT.

Further, since the determination as to whether or not the pixel of interest is a rail candidate point is made based on the minimum value and the maximum value of the evaluation value of luminance continuity, the extraction of the rail candidate point is performed at high speed. Can be done with

Further, by setting the small area for evaluating the continuity of luminance based on the rail width on the image, the extraction accuracy of the rail candidate points can be improved.

Further, the continuity of luminance at the extracted rail candidate point can be easily obtained based on the evaluation value of continuity of luminance. In other words, based on the evaluation value of the continuity of the luminance, the continuation direction of the rail when the rail candidate point is a rail can be easily obtained.

When detecting the rail reference point in detecting the traveling rail of the own vehicle, when detecting the rail reference point, each rail candidate point having the same luminance continuous direction as the rail candidate point in the reference point search area set on the image. Since the straight line detection process is performed and the rail reference point is detected based on the detected straight line, the accuracy of the rail reference point detection can be improved. That is, from the detected straight line group, a pair of straight lines whose distances are parallel to each other while corresponding to the actual distance between rails on the image are selected, and the rail reference point is detected based on these straight line pairs. The detection accuracy of the rail reference point can be improved.

By setting the reference point search area in front of the vehicle, the detection accuracy of the rail reference point can be improved, and the detection accuracy of the rail representative point connected to the rail reference point can be improved. Can be.

Further, when detecting the rail reference point, by excluding the rail candidate points corresponding to the three-dimensional object having the predetermined height from the processing targets, it is possible to increase the processing speed and improve the detection accuracy.

Further, when detecting the traveling rail of the own vehicle, the tracking of the rail representative point can be performed efficiently by performing the search for each search line set at predetermined intervals. Further, by setting the interval between the search lines to be narrower as the distance from the host vehicle becomes farther, the tracking of the rail representative point becomes more efficient.

In tracking the rail representative point, the predicted position of the rail representative point on the next search line is calculated based on the rail representative point detected in front, and each rail search on the search line for this predicted position is performed. By evaluating the degree of coincidence of the point positions, the detection accuracy of the rail representative point can be improved.

In tracking a rail representative point, a luminance pattern of a set area centered on a rail representative point detected in the front and a luminance pattern of a set area centered on each rail search point on the next search line. By evaluating the degree of coincidence with, the detection accuracy of the rail representative point can be improved.

In tracking the rail representative point, the degree of coincidence between the direction from the rail representative point detected in front to each rail search point on the next search line and the continuous direction of luminance at each rail search point is determined. By performing the evaluation, the detection accuracy of the rail representative point can be improved.

Further, when tracking the rail representative points, the detection accuracy of the rail representative points can be improved by complementing the left and right rail representative points with each other.

Further, when tracking the rail representative points, pixels corresponding to a three-dimensional object having a predetermined height and rail candidate points are excluded from processing targets, so that processing can be speeded up and detection accuracy can be improved. .

Further, by mounting the track recognition device 2 configured as described above on a railway vehicle 1 running on a business main line or the like after the end of commercial operation such as a maintenance vehicle, the recognized track information can be obtained. It can be used effectively and effectively.

In the present embodiment, an example of recognizing the own vehicle trajectory by detecting a pair of rails has been described. However, the present invention is not limited to this, and the present invention is not limited to this. Another elongated fixed object may be provided.

Further, in the present embodiment, the brightness data and the distance data in the scenery in the traveling direction of the own vehicle are acquired by using the stereo camera. However, the present invention is not limited to this. The luminance data and the distance data may be respectively obtained by the combination of the radars.

[0112]

As described above, according to the present invention, the trajectory is recognized based on the evaluation of the continuity of the luminance on the image, so that the trajectory can be accurately recognized without being affected by external light or the like. be able to.

[Brief description of the drawings]

FIG. 1 is a functional block diagram showing a track recognition device mounted on a track vehicle.

FIG. 2 is a flowchart of rail candidate point extraction processing;

FIG. 3 is an explanatory diagram showing a search area on an image.

FIG. 4 is an explanatory diagram showing a small area when evaluating the continuity of luminance in each search direction.

FIG. 5 is a table showing a relationship between a search direction of luminance continuity and its evaluation value.

FIG. 6 is an explanatory diagram showing a direction image of a rail candidate point.

FIG. 7 is a flowchart of a reference point detection process.

FIG. 8 is an explanatory diagram showing a reference point search area in a direction image.

FIG. 9 is a flowchart of rail tracking processing (part 1).

FIG. 10 is a flowchart of rail tracking processing (part 2).

FIG. 11 is an explanatory diagram showing a search line for a rail representative point.

[Explanation of symbols]

 REFERENCE SIGNS LIST 1 vehicle (railroad vehicle) 2 track recognition device 10 stereo camera (imaging means) 35 three-dimensional object detection unit (three-dimensional object detection unit) 40 rail candidate point extraction unit (fixed object candidate point extraction unit) 50 own vehicle traveling rail detection unit (Fixed object detecting means) 51 Reference point detecting unit (Reference point detecting means) 52 Rail tracking unit (Fixed object tracking means) 100 Control unit (Control means)

Claims (22)

[Claims] 1. An image of a traveling direction of a host vehicle captured by an image pickup means is used to detect a fixed object for running the host vehicle continuously disposed along the track and recognize the track of the host vehicle. A trajectory recognition device that performs fixed object candidate point extraction means for evaluating continuity of luminance at each pixel on the image to extract fixed object candidate points; A fixed object detecting means for detecting a fixed object disposed on a track of the vehicle based on the fixed object. 2. The fixed object candidate point extracting means evaluates continuity of luminance in a plurality of directions for each pixel of interest, and based on the evaluation value of the continuity of luminance in the plurality of directions. The trajectory recognition apparatus according to claim 1, wherein it is determined whether or not the target pixel is a fixed object candidate point. 3. The fixed object candidate point extracting means sets small areas of pixels connected in a plurality of different search directions with respect to a target pixel, and calculates a difference value of luminance in each of the small areas with the target pixel. The trajectory recognition device according to claim 1 or 2, wherein the evaluation value is an evaluation value of continuity of luminance in each search direction. 4. The fixed object candidate point extracting means sets small areas of pixels connected in a plurality of different search directions with respect to a target pixel, and calculates a variance value of luminance in each of the small areas by the target pixel. The trajectory recognition device according to claim 1 or 2, wherein the evaluation value is an evaluation value of continuity of luminance in each search direction. 5. The fixed object candidate point extracting means sets a small area of pixels connected in a plurality of different search directions with respect to a pixel of interest, and sets a difference value and a variance value of luminance in each of the small areas. 3. The trajectory recognition device according to claim 1, wherein a larger one of the trajectories is used as an evaluation value of continuity of luminance of the pixel of interest in each search direction. 6. The trajectory according to claim 3, wherein a length of the small area in a search direction is set longer than a width of the fixed object on an image. Recognition device. 7. The fixed object candidate point extracting means obtains a minimum value and a maximum value from among the evaluation values of the continuity of luminance of the pixel of interest in a plurality of directions, and calculates the evaluation values of these continuity of luminance. 7. The trajectory recognition device according to claim 2, wherein whether or not the pixel of interest is a fixed object candidate point is determined based on the minimum value and the maximum value. 8. The fixed object candidate point detecting means, based on a minimum value of the evaluation value of the luminance continuity and a difference between a maximum value and a minimum value of the evaluation value of the luminance continuity. The trajectory recognition device according to claim 7, wherein it is determined whether or not the target pixel is a fixed object candidate point. 9. The fixed object candidate point extracting means obtains a luminance continuous direction at the extracted fixed object candidate point based on an evaluation value of the luminance continuity to the target pixel. The trajectory recognition device according to any one of claims 2 to 8, wherein 10. A three-dimensional object detecting means for detecting a three-dimensional object in a traveling direction of the own vehicle, wherein said fixed object detecting means has a predetermined height range based on three-dimensional object information detected by said three-dimensional object detecting means. The solid object is detected, and the pixels and the fixed object candidate points corresponding to the solid object in the predetermined height range are excluded from the fixed object detection processing target. Item 10. The trajectory recognition device according to any one of Items 9. 11. A fixed object detecting means for detecting a fixed object reference point serving as a reference for fixed object detection based on the fixed object candidate points, the fixed object detecting means, 2. A fixed object tracking means for sequentially detecting fixed object representative points connected to a reference point.
The trajectory recognition device according to claim 10. 12. The reference point detection means sets a reference point search area in front of the vehicle, and determines the fixed object reference point based on the fixed object candidate points distributed in the reference point search area. The trajectory recognition device according to claim 11, wherein the trajectory is detected. 13. The fixed point detecting means performs a straight line detection process on the fixed object candidate points distributed in the reference point search area for each of the fixed object candidate points having the same luminance continuous direction. 13. The trajectory recognition device according to claim 12, wherein the fixed object reference point is detected based on the detected straight line. 14. The fixed object tracking means sets a search line at predetermined intervals on the image, and detects a fixed object search point within a predetermined range set on the search line in front of the fixed line. 12. The method according to claim 11, wherein the evaluation is performed based on the reference point of the object or the representative point of the fixed object, and the next representative point of the fixed object is detected.
The trajectory recognition device according to claim 13. 15. The trajectory recognition device according to claim 14, wherein an interval between the search lines is set to be narrower as the distance from the own vehicle increases. 16. The fixed object tracking means calculates a predicted position of the fixed object representative point on the next search line based on the fixed object reference point or the fixed object representative point detected in the foreground. 15. The evaluation index is a degree of coincidence of each fixed object search point on the search line with the predicted position.
Or the trajectory recognition device according to claim 15. 17. The fixed object tracking means includes: a luminance pattern of a setting area centered on a fixed object representative point detected in the foreground; and a setting pattern centered on each fixed object search point on a next search line. 17. The trajectory recognition device according to claim 14, wherein a degree of coincidence with the luminance pattern of the region is used as an evaluation index. 18. The fixed object tracking means, wherein: a direction of each fixed object search point on a next search line with respect to a fixed object representative point detected in the foreground; 18. The trajectory recognition device according to claim 14, wherein a degree of coincidence with a direction is used as an evaluation index. 19. The fixed object is a pair of left and right rails, wherein the reference point detecting means detects a pair of rail reference points as the fixed object reference point, and the fixed object tracking means comprises a pair of rails. 19. The trajectory recognition device according to claim 11, wherein a rail representative point connected to a rail reference point is sequentially detected as the fixed object representative point. 20. The trajectory recognition according to claim 19, wherein said reference point detecting means detects said pair of rail reference points based on a theoretical distance between rails obtained from an actual distance between rails. apparatus. 21. The trajectory recognition device according to claim 19, wherein the fixed object tracking means complements left and right rail representative points with each other. 22. A railway vehicle using the track recognition device according to any one of claims 1 to 21, wherein a vehicle control or a track to a passenger is performed based on the track information recognized by the track recognition device. A railway vehicle using a track recognizing device, comprising: control means for performing at least one of display control of information.