JP2014062415A - Trajectory detector and trajectory monitoring device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a trajectory detector capable of reducing processing time and reducing erroneous identification of a trajectory, and a trajectory monitoring device with the trajectory detector.SOLUTION: A trajectory detection follow-up part calculates a left rail approximate curve so as to pass through a lower end L of a left rail, and vanishing points V of the left rail and a right rail in frame image data 20, and calculates a right rail approximate curve so as to pass through a lower end R and the vanishing point V of the right rail in the frame image data 20 when the first frame image data 20 is input from an image photographing device. In addition, the trajectory detection follow-up part fixes the lower end L of the left rail and the lower end R of the right rail R, and calculates the left rail approximate curve and the right rail approximate curve in frame image data after the first frame image data 20.

Description

  The present invention analyzes a traveling direction image captured by an image capturing device mounted on a vehicle traveling on a track, and detects a position of the track existing in front of the vehicle, and the track detection device. And a trajectory monitoring device.

  The train forward railway signal / sign recognition device described in Patent Document 1 includes a video camera and a railway signal / sign recognition device, and performs pattern matching to recognize the railway signal / signs installed near the rail ahead of the train. In the train ahead railway signal / sign recognition device, it corresponds to the means for extracting the rail in the search screen, which is a one-frame screen obtained from the captured image data ahead of the train, and the scale of the template image changed in stages. Corresponding to search area specifying means for specifying the search area in the vicinity of the rail on the search screen using information specific to the railway, means for performing pattern matching for each search area, and pattern matching that gave the highest score It comprises a means for determining a search area and recognizing a railway signal / sign. Therefore, it is possible to recognize railway signals and signs existing in the vicinity of the rail.

JP 2008-215938 A (5 pages 23 lines to 8 pages 4 lines) JP 2010-286985 A

King Hann Lim et al., "Lane Detection and Kalman-based Linear-Parabolic Lane Tracking", 2009 International Conference on Intelligent Human-Machine Systems and Cybernetics, pp, 351-354, August 2009. Richard O. Duda et al., "Use of the Hough Transformation to Detect Lines and Curves in Pictures", Communications of the ACM, Vol. 15, No. 1, pp. 11-15, January 1972. Herve Abdi, "A Neural Network Primer", Journal of Biological Systems, Vol.2, No. 3, pp. 247-283, 1994. Lawrence R. Rabiner, "A Tutorial on Hidden Markov Models and Selected Applications in Speech Recognition", Proceedings of the IEEE, Vol. 77, No. 2, pp. 257-286, February 1989.

  The train front railway signal / sign recognition device according to the prior art takes a relatively long time for two-dimensional image processing, and when the image quality deteriorates due to weather and sunshine conditions, the rail cannot be detected stably. Further, since the rail is detected based only on the image of one frame, there is a possibility that the rail is erroneously identified by chance.

  An object of the present invention is to provide a trajectory detection device that solves the above-described problems, reduces processing time and reduces misidentification of trajectories, and a trajectory monitoring device including the trajectory detection device. There is to do.

  The track detection device according to the present invention is fixed to a train traveling on a track including a left rail and a right rail, acquires moving image data of the traveling direction of the train, and obtains a plurality of frame image data included in the moving image data. Image capturing means for sequentially outputting, trajectory detection tracking means for calculating a left rail approximate curve representing the position of the left rail in each frame image data, and a right rail approximate curve representing the position of the right rail. In the trajectory detection device, the trajectory detection follow-up means receives the first frame image data from the image photographing means, the lower end of the left rail, the left rail and the right in the first frame image data. The left rail approximate curve is calculated so as to pass through the vanishing point of the rail, and the lower and upper ends of the right rail in the first frame image data are calculated. When the right rail approximate curve is calculated so as to pass through the vanishing point, and the second frame image data after the first frame image data is input from the image photographing means, the first frame image data The left rail approximate curve in the second frame image data is calculated so as to pass through the lower end of the left rail, and the second rail is passed through the lower end of the right rail in the first frame image data. The right rail approximate curve in the frame image data is calculated.

  According to the trajectory detection device and the trajectory monitoring device according to the present invention, the trajectory detection follow-up means receives the first frame image data from the image photographing means, the lower end of the left rail in the first frame image data, and the left The left rail approximate curve is calculated so as to pass through the vanishing point of the rail and the right rail, and the right rail approximate curve is calculated so as to pass through the lower end of the right rail and the vanishing point in the first frame image data. In addition, when the second frame image data after the first frame image data is input from the image capturing unit, the trajectory detection follow-up unit passes through the lower end of the left rail in the first frame image data. The left rail approximate curve in the second frame image data is calculated so as to pass through the lower end of the right rail in the first frame image data. Therefore, it is possible to reduce processing time and to reduce misidentification of the trajectory as compared with the prior art.

It is a block diagram which shows the structure of the orbit detection apparatus 100 which concerns on Embodiment 1 of this invention. It is a flowchart which shows the track | orbit detection follow-up process performed by the track | orbit detection follow-up part 2 of FIG. It is a flowchart which shows the left rail detection follow-up process performed in step S2 of FIG. An example of the frame image data 20 output from the image photographing device 1 of FIG. 1, the initial position of the vanishing point V obtained by executing the process of step S1 of FIG. 2 on the frame image data 20, and the left It is a photograph which shows the position of the rail lower end L, and the position of the right rail lower end R. It is a graph which shows the left rail approximated curve 10 obtained in step S11 of FIG. 4 is a graph showing a left rail approximate curve 29 when a control point 28 is used in place of the control point 27 in step S11 of FIG. It is a graph which shows the five rectangular area | regions 24 set in step S12 of FIG. 4 is a graph showing a line segment 24S detected in each rectangular area 24 in step S14 of FIG. It is a graph which shows the left rail approximated curve 26 calculated in step S15 of FIG. It is a block diagram which shows the structure of the track monitoring apparatus 101 which concerns on Embodiment 2 of this invention. The remaining snow detection area 14 set by the remaining snow detection area setting unit 4 in FIG. 10, the image data 15 in the remaining snow detection area 14, the normalized rectangular image data 16 generated based on the image data 15, and the normalized rectangle It is explanatory drawing which shows the one-dimensional projection data 17 and 21 produced | generated based on the image data 16. FIG. It is explanatory drawing which shows the structure and operation | movement of the remaining snow existence estimated value calculation part 6 of FIG. It is a block diagram which shows the structure of the time series determination part 7 of FIG. It is a graph which shows an example of the time series data 26 of the remaining snow existence estimated value 25 of FIG. 10 at the time of low speed. It is a graph which shows an example of the time series data 26 of the remaining snow existence estimation value 25 of FIG. 10 at the time of high speed. It is a block diagram which shows the structure of 101 A of track monitoring apparatuses which concern on Embodiment 3 of this invention. It is a graph which shows the remaining snow detection check range 22 used by 6 A of remaining snow existence estimated value calculation parts of FIG. 16, and the one-dimensional projection data 21 when there is remaining snow. It is a graph which shows the remaining snow detection check range 22 used by 6A of remaining snow existence estimated value calculation parts of FIG. 16, and the one-dimensional projection data 21 when there is no remaining snow. It is a block diagram which shows the structure of the trajectory monitoring apparatus 101B which concerns on Embodiment 4 of this invention. It is a block diagram which shows the structure of the time series determination part 7A of FIG. It is explanatory drawing which shows operation | movement of the time series determination part 7A of FIG.

  Hereinafter, embodiments according to the present invention will be described with reference to the drawings. In addition, in each following embodiment, the same code | symbol is attached | subjected about the same component.

Embodiment 1 FIG.
FIG. 1 is a block diagram showing a configuration of a trajectory detection apparatus 100 according to Embodiment 1 of the present invention. The track detection device 100 is mounted on a cab of a train that runs on two rails, and includes an image capturing device 1, a track detection tracking unit 2, and an output unit 3. Here, each function of the trajectory detection follow-up unit 2 and the output unit 3 is realized by, for example, a CPU (Central Processing Unit) of the trajectory detection device 100 executing a program corresponding to each function. The image capturing device 1 is an image capturing device such as a video camera installed in a traveling direction on a cab provided at the head of a train, and captures a left rail and a right rail lying in the traveling direction of the train. Then, the frame image data 20 included in the captured moving image data is sequentially output to the trajectory detection follow-up unit 2.

  Next, the operation of the trajectory detection follow-up unit 2 will be described. FIG. 2 is a flowchart showing the trajectory detection tracking process executed by the trajectory detection tracking unit 2 of FIG. First, in step S1 of FIG. 2, the trajectory detection follow-up unit 2 sets the initial position of the vanishing point V of the two rails and the position of the lower end L of the left rail in the first frame image data 20 from the image capturing device 1. And the position of the lower end R of the right rail. The processing executed in step S1 is specifically as follows. The relative position of the train with respect to the two rails does not change, and in particular, the position of the left rail lower end L and the position of the right rail lower end R in the frame image data 20 do not substantially change. Using this, the trajectory detection follow-up unit 2 uses, for example, the three-dimensional position information of the image capturing device 1, the three-dimensional vector information of the visual axis indicating the capturing direction of the image capturing device 1, and the image capturing devices 1 and 2. Based on the distance along the traveling direction between the two rails and the distance along the axis perpendicular to the traveling direction between the imaging device 1 and the two rails on the frame image data 20 The position of the left rail lower end L and the position of the right rail lower end R are calculated and acquired. Further, the trajectory detection follow-up unit 2 detects a line segment on the frame image data 20 such as a Hough transform, based on the calculated position of the left rail lower end L and the position of the right rail lower end R. The intersection point on the extension of the two estimated line segments is acquired as the initial position of the vanishing point V. Note that the initial position of the vanishing point V, the position of the left rail lower end L, and the position of the right rail lower end R in the first frame image data 20 may be determined and obtained by visual judgment by a human.

  As described above, the position of the left rail lower end L and the position of the right rail lower end R in the frame image data 20 are not substantially changed during the traveling of the train. Therefore, the trajectory detection follow-up unit 2 uses the position of the left rail lower end L and the position of the right rail lower end R acquired in step S1 without changing in each process after step S2 of the trajectory detection follow-up process.

  FIG. 4 shows an example of the frame image data 20 output from the image capturing device 1 of FIG. 1 and the initial vanishing point V acquired by executing the processing of step S1 of FIG. It is a photograph which shows a position, the position of the left rail lower end L, and the position of the right rail lower end R. As shown in FIG. 4, an xy coordinate system is defined in which the lower left corner of the frame image data 20 is the origin O. In FIG. 2, in step S <b> 2 following step S <b> 1, the trajectory detection tracking unit 2 calculates the left rail detection tracking process of FIG. 3 for calculating the left rail approximate curve 10 and the right rail approximate curve 11. The right rail detection tracking process is executed, and the trajectory detection tracking process is terminated. Since the right rail detection tracking process is the same as the left rail detection tracking process, only the left rail detection tracking process will be described below.

  FIG. 3 is a flowchart showing the left rail detection follow-up process executed in step S2 of FIG. In step S11 of FIG. 3, the trajectory detection follow-up unit 2 determines the control point 27, calculates a quadratic spline curve that passes through the vanishing point V, the left rail lower end L, and the control point 27. The next spline curve is set to the left rail approximate curve 10. Specifically, the track detection follow-up unit 2 performs the process of step S11 using the lane detection method described in Non-Patent Document 1. FIG. 5 is a graph showing the left rail approximate curve 10 obtained in step S11 of FIG. 3, and FIG. 6 shows the left rail when the control point 28 is used in place of the control point 27 in step S11 of FIG. 3 is a graph showing an approximate curve 29. As shown in FIGS. 5 and 6, the secondary spline curve can be changed by moving the control point from the control point 27 to the control point 28, so that the vanishing point V and the left rail lower end L are passed. Arbitrary curves can be obtained.

  3, in step S12 following step S11, the trajectory detection follow-up unit 2 equally divides the region between the x axis and the horizontal line 30 including the vanishing point V into five regions parallel to the x axis. In each region, a rectangular region 24 including the left rail approximate curve 10 is set. FIG. 7 is a graph showing the five rectangular areas 24 set in step S12 of FIG. As shown in FIG. 7, each rectangular area 24 has a predetermined width W <b> 24 and is set so that the left rail approximate curve 10 passes through the center 24 </ b> C of the rectangular area 24. Here, the width W24 is set to, for example, 25% of the width of the frame image data 20 in the x-axis direction. In the following steps S14 and S15, the five rectangular areas 24 are used to detect the position of the left rail that changes as the train progresses on the frame image data 20 that is input next.

  In step S13 following step S12 in FIG. 3, the trajectory detection follow-up unit 2 determines whether or not the next frame image data 20 has been input from the image capturing apparatus 1 within a predetermined time. While the process proceeds to S14, when the determination is NO, the process returns to the trajectory detection tracking process of FIG. Here, the predetermined time is set, for example, as a generation time interval of the frame image data 20 in the image capturing apparatus 1. In step S14, the trajectory detection follow-up unit 2 detects a line segment 24S corresponding to the left rail using the Hough transform in each rectangular area 24 of the input frame image data 20. Specifically, the trajectory detection follow-up unit 2 executes the process of step S14 using the method described in Non-Patent Document 2. FIG. 8 is a graph showing the line segment 24S detected in each rectangular area 24 in step S14 of FIG.

  In step S15 following step S14 in FIG. 3, the trajectory detection follow-up unit 2 applies a spline curve passing through the lower end L of the left rail to the five line segments 24S of each rectangular region 24, thereby approximating the left rail. The curve 26 is calculated, the left rail approximate curve 10 is updated to the left rail approximate curve 26, and is output to the output unit 3. Specifically, the track detection follower 2 fixes the position of the lower end L of the left rail, and the vanishing point and control point of the left rail approximate curve 26 move from the vanishing point V and control point 27 of the left rail approximate curve 10. Under the condition that it is possible, the left rail approximation is performed so as to minimize the sum of the magnitudes of errors between the x coordinate of the left rail approximate curve 26 and the x coordinate of each line segment 24S at each y coordinate position. A curve 26 is calculated. FIG. 9 is a graph showing the left rail approximate curve 26 calculated in step S15 of FIG. Furthermore, in step S <b> 14, the output unit 3 outputs the updated left rail approximate curve 10 to, for example, a display and displays it together with the frame image data 20.

  Next, in step S16, the trajectory detection follow-up unit 2 changes the area between the x-axis and the horizontal line 30 including the vanishing point V of the updated left rail approximate curve 10 into five areas parallel to the x-axis. Divide into equal parts, set a rectangular area 24 including the left rail approximate curve 10 in each area, and return to step S13. Note that the method for setting the rectangular area 24 in step S16 is the same as the setting method in step S12. Further, the vanishing point V of the updated left rail approximate curve 10 used in step S16 is the same as the right rail approximate curve 11 calculated in the same manner as the left rail approximate curve 10 in the frame image data 20 from which the left rail approximate curve 10 is calculated. The intersection with the left rail approximate curve 10.

  As described above, the track detection device 100 according to the present embodiment is fixed to a train traveling on a track including the left rail and the right rail, acquires moving image data of the traveling direction of the train, and is included in the moving image data. The image capturing device 1 that sequentially outputs a plurality of frame image data 20, the left rail approximate curve 10 that represents the position of the left rail in each frame image data 20, and the right rail approximate curve 11 that represents the position of the right rail are calculated. The trajectory detection follow-up unit 2 is configured. Here, when the first frame image data 20 is input from the image capturing device 1, the trajectory detection follow-up unit 2 includes the lower end L of the left rail and the vanishing points V of the left rail and the right rail in the first frame image data 20. The left rail approximate curve 10 is calculated so as to pass through, and the right rail approximate curve 11 is calculated so as to pass through the lower end R and vanishing point V of the right rail in the first frame image data 20. Furthermore, when the trajectory detection follow-up unit 2 inputs the frame image data 20 after the first frame image data from the image capturing device 1, the input frame passes through the lower end L of the left rail in the first frame image data. The left rail approximate curve 10 in the image data 20 is calculated, and the right rail approximate curve 11 in the input frame image data 20 is calculated so as to pass through the lower end R of the right rail in the first frame image data 20.

  In addition, when the frame image data 20 after the first frame image data 20 is input, the trajectory detection follow-up unit 2 calculates the left rail approximate curve calculated in the frame image data 20 immediately before the input frame image data 20. By detecting the line segment 24S in the predetermined region 24 including 10, the left rail approximate curve 10 in the input frame image data 20 is calculated, and in the frame image data 20 immediately before the input frame image data 20 By detecting a line segment 24S in a predetermined region 24 including the calculated right rail approximate curve 11, the right rail approximate curve 11 in the input frame image data 20 is calculated.

  Therefore, according to the present embodiment, the position of the left rail lower end L and the position of the right rail lower end R in the frame image data 20 are acquired in advance, and the position of the left rail lower end L and the position of the right rail lower end in each frame image data 20 are acquired. Since the left rail and the right rail are detected while fixing the position of R, the processing time can be reduced as compared with the prior art. Further, even if the contrast of the frame image data 20 decreases due to the influence of weather and sunshine conditions, the left rail and the right rail are detected more stably than in the prior art, and the detected positions of the left rail and the right rail are detected. Can follow the positions of the left rail and the right rail on the frame image data 20. Therefore, it is possible to reduce misidentification of the left rail and the right rail as compared with the prior art.

  In addition, since the train ahead railway signal / sign recognition device described in Patent Document 1 detects a rail based only on an image of one frame, there is a possibility that the rail is accidentally mistakenly identified. For example, since the position of the rail in the frame image data 20 to be processed is detected using the rectangular area 24 set in the frame image data 20 immediately before the frame image data 20 to be processed, misidentification can be reduced. .

  In the present embodiment, each function of the trajectory detection follow-up unit 2 and the output unit 3 is realized by executing a program corresponding to each function, for example, by the CPU of the trajectory detection device 100. However, the present invention is not limited thereto, and may be realized by configuring a hardware circuit corresponding to each function.

  Further, the trajectory detection follow-up unit 2 divides the region between the x axis and the horizontal line 30 including the vanishing point V into five regions parallel to the x axis, and includes the left rail approximate curve 10 in each region. Although the rectangular area 24 is set, the present invention is not limited to this. The trajectory detection follow-up unit 2 divides the region between the x-axis and the horizontal line 30 including the vanishing point V into two or more regions parallel to the x-axis, and in each region, the left rail approximate curve 10 is The rectangular area 24 to be included may be set.

Embodiment 2. FIG.
FIG. 10 is a block diagram showing the configuration of the trajectory monitoring apparatus 101 according to Embodiment 2 of the present invention. The trajectory monitoring apparatus 101 includes a trajectory detection apparatus 100 according to the first embodiment, a remaining snow detection area setting unit 4, a one-dimensional projection data generation unit 5, a remaining snow presence / absence estimated value calculation unit 6, and a time series determination unit 7. The vehicle speed detection part 8 and the output part 9 are comprised, and the presence or absence of the remaining snow between two rails is determined. Here, in the present embodiment, the trajectory detection device 100 operates in the same manner as the trajectory detection device 100 according to the first embodiment, and the position of the vanishing point V, the position of the left rail lower end L, and the right rail lower end R. , The left rail approximate curve 10 and the right rail approximate curve 11 are output to the remaining snow detection area setting unit 4. The remaining snow detection area setting unit 4, the one-dimensional projection data generation unit 5, the remaining snow presence / absence estimated value calculation unit 6, the time series determination unit 7, the vehicle speed detection unit 8, and the output unit 9 are For example, the program corresponding to each function is executed by the CPU of the trajectory monitoring apparatus 101.

  With reference to FIG. 11, operations of the remaining snow detection region setting unit 4 and the one-dimensional projection data generation unit 5 in FIG. 10 will be described. 11 shows the remaining snow detection area 14 set by the remaining snow detection area setting unit 4 in FIG. 10, the image data 15 in the remaining snow detection area 14, and the normalized rectangular image data 16 generated based on the image data 15. FIG. 4 is an explanatory diagram showing one-dimensional projection data 17 and 21 generated based on normalized rectangular image data 16.

  First, as shown in FIG. 11, the remaining snow detection region setting unit 4 passes through a position Ls that is a predetermined distance ws away from the position of the lower end L of the left rail in the negative direction of the x axis and the vanishing point V. A secondary spline curve is calculated as the left rail road shoulder curve 12. Further, a quadratic spline curve passing through a position Rs that is a predetermined distance ws away from the position of the right rail lower end R in the positive direction of the x-axis and the vanishing point V is calculated as the right rail road shoulder curve 13. Here, the distance ws is set to a distance corresponding to the width of the shoulder of the left rail and the right rail on the x-axis of the frame image data 20, and the left rail shoulder curve 12 and the right rail shoulder approximate curve 13 are And corresponding to the outer shoulder edge of the right rail.

  Next, the remaining snow detection area setting unit 4 sets the remaining snow detection area 14 for detecting the presence or absence of remaining snow to a trapezoid area having vertices L1, L2, R1, and R2, and image data 15 in the remaining snow detection area 14 is set. Is output to the one-dimensional projection data generation unit 5. Here, the y coordinates of the vertices L1 and R1 are yh × a (where yh is the y coordinate of the horizontal line 30 and 0 <a <1), and the y coordinates of the vertices L2 and R2 are yh × b. (However, 0 <b <a <1.) Further, the side L1-L2 is a line segment that approximates the left rail road shoulder curve 12, and the side R1-R2 is a line segment that approximates the right rail road shoulder curve 13. The constant a is set to 0.6, for example, and the constant b is set to 0.4, for example. The remaining snow detection area setting unit 4 set as described above includes a part of the left rail approximate curve 10 and a part of the right rail approximate curve 11.

  As shown in FIG. 11, the one-dimensional projection data generation unit 5 geometrically converts the image data 15 having a trapezoidal shape into image data having a rectangular shape, and normalizes the luminance value of the image data 15 after the geometric conversion. Conversion to standardized image data 16 is performed. Specifically, the one-dimensional projection data generation unit 5 linearly converts the luminance value so that the minimum value of the luminance value of the image data after geometric conversion becomes 0 and the maximum value becomes 255, for example. Note that, as shown in FIG. 11, a pq coordinate system is defined in which the lower left corner of the normalized image data 16 is the origin O. At this time, the position of the left rail lower end L and the position of the right rail lower end R in the frame image data 20 are converted into the position 18 and the position 19 on the p-axis in the normalized image data 16, respectively. Next, the one-dimensional projection data generation unit 5 generates the one-dimensional projection data 17 by projecting the standardized image data 16 in the q-axis direction. Specifically, the one-dimensional projection data generation unit 5 generates the one-dimensional projection data 17 by accumulating pixel values in the q-axis direction at each position on the p-axis. Further, the one-dimensional projection data generation unit 5 uses only the data between the left rail position 18 and the right rail position 19 in the one-dimensional projection data 17 as one-dimensional projection data 21 having a plurality of N pieces of data. Output to remaining snow presence / absence estimated value calculation unit 6. The one-dimensional projection data 17 and 21 generated as described above indicate the luminance distribution along the direction crossing the left rail and the right rail.

  FIG. 12 is an explanatory diagram showing the configuration and operation of the remaining snow presence / absence estimated value calculation unit 6 of FIG. 12, the remaining snow presence / absence estimated value calculation unit 6 includes an input layer 61 having N neurons, an intermediate layer 62 having a plurality of neurons, and an output layer 63 having a single neuron. A layered neural network and a learning controller 60 are provided. The one-dimensional projection data 21 from the one-dimensional projection data generation unit 5 is input to the input layer 61, and is input to the intermediate layer 62 according to each load coefficient between each neuron of the input layer 61 and each neuron of the intermediate layer 62. After being output, the remaining snow presence / absence estimated value 25 is output from the output layer 63 according to each load coefficient between each neuron of the intermediate layer 62 and the neuron of the output layer 63.

  Further, the learning controller 60 inputs the learning data input value of the one-dimensional projection data 21 when there is residual snow between the left rail and the right rail, and one-dimensional when there is no residual snow between the left rail and the right rail. Using the learning data input value of the projection data 21, when there is remaining snow between the left rail and the right rail, the remaining snow presence / absence estimated value 25 becomes 1, and there is no remaining snow between the left rail and the right rail. Sometimes, the load coefficient between each neuron in the input layer 61 and each neuron in the intermediate layer 62 and each neuron in the intermediate layer 62 and each neuron in the output layer 63 are set so that the remaining snow presence / absence estimated value 25 becomes zero. Each load coefficient in between is learned and determined in advance. Therefore, the remaining snow presence / absence estimated value 25 from the output layer 63 is close to 1 when there is remaining snow, and close to 0 when there is no remaining snow. Note that the learning controller 60 determines each load coefficient using, for example, a back propagation learning rule using a three-layer perceptron described in Non-Patent Document 3. The learning data includes one-dimensional projection data 21 under a plurality of weather conditions and a plurality of solar radiation conditions.

  FIG. 13 is a block diagram showing a configuration of the time series determination unit 7 of FIG. In FIG. 13, a time series determination unit 7 includes a three-layer structure low-speed determination neural network 7L, a three-layer structure high-speed determination neural network 7H, learning controllers 70 and 74, a remaining snow determination unit 78, A neural network selection unit 79 and a switch SW are provided. Further, the low-speed determination neural network 7L includes an input layer 71 having a plurality of neurons, an intermediate layer 62 having a plurality of neurons, and an output layer 73 having a single neuron. The neural network 7H includes an input layer 75 including a plurality of neurons, an intermediate layer 76 including a plurality of neurons, and an output layer 74 including a single neuron.

  In FIG. 13, the neural network selection unit 79 determines whether or not the train speed is equal to or higher than a predetermined speed threshold based on the train speed information from the vehicle speed detection unit 8. When the speed is less than the predetermined speed threshold, the switch SW is switched to the low-speed determination neural network 7L side for a predetermined period T, while when the train speed is equal to or higher than the predetermined speed threshold, the switch SW is Is switched to the high-speed determination neural network 7H side during the period T. Here, the vehicle speed detection unit 8 is based on, for example, a vehicle speed sensor that calculates the speed of the train based on the number of rotations of the wheels or the position information of the train from a GPS (Global Positioning System) that can be used outdoors other than the tunnel. This is a vehicle speed calculation device that calculates the speed of the train and outputs it to the neural network selection unit 79. The switch SW converts the time series data 26 within the predetermined period T of the remaining snow presence / absence estimated value 25 from the remaining snow presence / absence estimated value calculation unit 6 to the input layer 71 of the low speed determination neural network 7L or the high speed determination neural network 7H. Are output to the input layer 75.

  When the time series data 26 is input to the input layer 71, the time series data 26 is output to the intermediate layer 71 in accordance with each load coefficient between each neuron of the input layer 71 and each neuron of the intermediate layer 71, and then the intermediate layer 71. Are output from the output layer 73 to the remaining snow determination unit 78 in accordance with the respective load coefficients between the respective neurons and the neurons of the output layer 73. Further, when the time series data 26 is input to the input layer 75, the time series data 26 is output to the intermediate layer 76 in accordance with each load coefficient between each neuron of the input layer 75 and each neuron of the intermediate layer 76, and then intermediate In response to each load coefficient between each neuron in the layer 76 and each neuron in the output layer 77, the remaining snow is determined from the output layer 76 to the remaining snow determination unit 78.

  The learning controller 70 also inputs the learning data input value of the time-series data 26 when the train speed is less than a predetermined speed threshold and there is residual snow between the left rail and the right rail, and the train speed. Is less than a predetermined speed threshold value, and the remaining snow between the left rail and the right rail using the input value of the learning data of the time series data 26 when there is no remaining snow between the left rail and the right rail. Each of the neurons of the input layer 71 and the intermediate layer 72 are set such that the remaining snow presence / absence determination value 32 becomes 1 and there is no remaining snow between the left rail and the right rail. Each load coefficient between each neuron and each load coefficient between each neuron in the intermediate layer 72 and each neuron in the output layer 73 is determined by learning in advance. Therefore, the output value from the output layer 73 is a value close to 1 when there is residual snow, and a value close to 0 when there is no residual snow.

  Further, the learning controller 74 inputs the learning data input value of the time-series data 26 when the train speed is equal to or higher than a predetermined speed threshold and there is residual snow between the left rail and the right rail, and the train speed. Remaining snow between the left rail and the right rail using the input value of the learning data of the time series data 26 when is equal to or greater than a predetermined speed threshold and there is no remaining snow between the left rail and the right rail. Each of the neurons in the input layer 75 and the intermediate layer 76 are set so that the remaining snow presence / absence determination value 32 becomes 1 when there is, and the remaining snow presence / absence determination value 32 becomes 0 when there is no remaining snow between the left rail and the right rail. Each load coefficient between each neuron and each load coefficient between each neuron in the intermediate layer 76 and each neuron in the output layer 77 is determined by learning in advance. Therefore, the output value from the output layer 77 is a value close to 1 when there is remaining snow, and a value close to 0 when there is no remaining snow.

  Note that the learning controller 60 uses, for example, a learning algorithm such as a back propagation learning rule using a three-layer perceptron described in Non-Patent Document 3 or a hidden Markov model described in Non-Patent Document 4 to load each weight coefficient. To decide.

  In FIG. 13, the remaining snow determination unit 78 determines that there is remaining snow when the output value from the output layer 73 or 77 is equal to or greater than a predetermined determination threshold, while the output value from the output layer 73 or 77 is When it is less than the predetermined remaining snow determination threshold, it is determined that there is no remaining snow, and a remaining snow presence / absence determination value 32 indicating the determination result is output to the output unit 9. Here, the remaining snow determination threshold value is set to 0.5, for example. Further, the output unit 9 outputs the remaining snow presence / absence determination value 32 to, for example, a display and displays it together with the frame image data 20.

As described above, the trajectory monitoring apparatus 101 according to the present embodiment is
(A) the trajectory detection device 100;
(B) In each frame image data 20, a remaining snow detection region setting for setting a detection target detection region 14 including a part of the left rail approximate curve 10 and a part of the right rail approximate curve 11 calculated by the trajectory detection follow-up unit 2. Part 4
(C) One-dimensional projection data generation for generating one-dimensional projection data 21 indicating the luminance distribution along the direction crossing the left rail and the right rail using the image data 15 in the remaining snow detection region 14 of each frame image data 20 Part 5;
(D) based on the one-dimensional projection data 21 of each frame image data 20, a remaining snow presence / absence estimated value calculation unit 6 that calculates a remaining snow presence / absence estimated value 25 indicating the presence or absence of remaining snow between the left rail and the right rail;
(E) Using the time series data 26 of the remaining snow presence / absence estimated value 25 calculated in each frame image data 20 and the train speed information, the presence / absence of remaining snow between the left rail and the right rail is determined, A time-series determination unit 7 that outputs the determination result is provided.

  Further, the remaining snow presence / absence estimated value calculation unit 6 inputs the one-dimensional projection data 21 when there is remaining snow between the left rail and the right rail, and 1 when there is no remaining snow between the left rail and the right rail. Each parameter (load coefficient) used to calculate the remaining snow presence / absence estimated value 25 is learned and determined in advance using the input value of the two-dimensional projection data 21, and is determined based on the one-dimensional projection data 21. The remaining snow presence / absence estimated value 25 is calculated using each parameter.

Furthermore, the time series determination unit 7
(A) Provided corresponding to each of a plurality of speed classifications of the train, and based on the input value of the time series data 26, using a predetermined load coefficient, indicates the presence or absence of residual snow between the left rail and the right rail A plurality of neural networks 7L and 7H for calculating output values;
(B) For each neural network 7L and 7H, the input value of the time-series data 26 when the train speed is the speed corresponding to the neural network and there is residual snow between the left rail and the right rail, The load coefficient of each neural network 7L and 7H is previously learned using the input value of the time series data 26 when the speed is a speed corresponding to the neural network and there is no remaining snow between the left rail and the right rail. Learning controllers 70 and 74 to be determined,
(C) The neural network 7L or 7H provided corresponding to the speed classification including the train speed is selected based on the train speed, and the time-series data 26 of the remaining snow presence / absence estimated value 25 is selected. A neural network selection unit 79 for outputting to 7L or 7H;
(D) A remaining snow presence / absence determining unit 78 that determines the presence / absence of remaining snow between the left rail and the right rail based on the output value from the selected neural network 7L or 7H is provided.

  As a simple method for determining the presence or absence of remaining snow between the left rail and the right rail, if the remaining snow presence / absence estimated value 25 from the remaining snow presence / absence estimated value calculation unit 6 is equal to or greater than a predetermined threshold, it is determined that there is remaining snow. There is a method of determining that there is no remaining snow if the remaining snow presence / absence estimated value 25 is less than a predetermined threshold. However, in the case of this method, if an image pattern similar to the remaining snow appears by chance due to the water pool between the rails or the sunshine condition, it is erroneously determined that there is remaining snow. In the present embodiment, since the presence / absence of remaining snow is determined using the time series data 26 of the remaining snow presence / absence estimated value 25, erroneous determination can be reduced.

  14 is a graph showing an example of the time series data 26 of the remaining snow presence / absence estimated value 25 of FIG. 10 at low speed, and FIG. 15 is an example of the time series data 26 of the remaining snow presence / absence estimated value 25 of FIG. 10 at high speed. It is a graph which shows. As shown in FIGS. 14 and 15, in general, when there is residual snow between the rails, a relatively large residual snow presence / absence estimated value 25 is output over a long period of time as the train speed decreases. For this reason, determining the presence or absence of residual snow using the same neural network without considering the train speed leads to an erroneous determination. In this embodiment, since the neural network 7H or 7L is selected and used according to the train speed, the presence or absence of remaining snow can be determined stably even if the train speed changes.

  Further, according to the present embodiment, the remaining snow presence / absence estimated value calculation unit 6 includes the neural network for calculating the remaining snow presence / absence estimated value 25 based on the one-dimensional projection data 21, and the load coefficient of the neural network is determined in advance. Learn and decide. Accordingly, even when only a relatively simple statistical parameter such as an average value or variance of the one-dimensional projection data 21 is compared with a predetermined threshold value and the presence or absence of remaining snow cannot be estimated, the presence or absence of remaining snow is automatically determined. it can. Furthermore, since learning is performed in advance using learning data including the one-dimensional projection data 21 under a plurality of weather conditions and a plurality of solar radiation conditions, it is not necessary for a human to manually extract the features of the one-dimensional projection data 21 and Misidentification can be reduced compared to the prior art.

  Furthermore, according to the present embodiment, the image capturing apparatus 1 captures an image in the train traveling direction to generate frame image data 20, and the trajectory detection follow-up unit 2 detects the two rails in the first frame image data 20. After calculating the left rail approximate curve 10 and the right rail approximate curve 11 based on the initial position of the vanishing point V, the position of the left rail lower end L, and the position of the right rail lower end R, the frame image data 20 sequentially input. Each time, the left rail approximate curve 10 and the right rail approximate curve 11 are updated. Further, the remaining snow detection area setting unit 4 sets a remaining snow detection area 14 necessary for determining the presence or absence of remaining snow between the left rail and the right rail, and acquires image data 15 in the remaining snow detection area 14. Further, the one-dimensional projection data generation unit 5 converts the image data 15 into one-dimensional projection data 17, and the remaining snow presence / absence estimated value calculation unit 6 uses the neural network (identification circuit) learned in advance to generate the one-dimensional projection data. A remaining snow presence / absence estimated value 25 is calculated and output based on a part of the one-dimensional projection data 21 of 17. Then, the time series determination unit 7 determines the presence / absence of remaining snow using the time series data 26 of the estimated remaining snow presence / absence value 25 and the train speed. Accordingly, it is possible to determine the presence or absence of remaining snow between the left rail and the right rail stably at a high speed, with less memory resources, and without depending on the shooting environment, as compared with the prior art.

  In the present embodiment, the train speed is classified into two stages, high speed and low speed, and the time-series determination unit 7 determines the low speed determination neural network 7L or the high speed determination neural network 7H according to the train speed. Although selected and used, the present invention is not limited to this. The train speed may be classified into three or more stages, and one of a plurality of neural networks may be selected and used according to the train speed.

  In the present embodiment, the neural network of the remaining snow presence / absence estimated value calculation unit 6 includes one intermediate layer 61. However, the present invention is not limited to this, and may include a plurality of intermediate layers. Further, the low-speed determination neural network 7L and the high-speed determination neural network 7H each include one intermediate layer 72 and 76, but the present invention is not limited to this, and may include a plurality of intermediate layers.

  Furthermore, in the present embodiment, the remaining snow detection area setting unit 4, the one-dimensional projection data generation unit 5, the remaining snow presence / absence estimated value calculation unit 6, the time series determination unit 7, the vehicle speed detection unit 8, and the output unit 9 Each function is realized by, for example, the CPU of the trajectory monitoring apparatus 101 executing a program corresponding to each function, but the present invention is not limited to this, and a hardware circuit corresponding to each function is configured. May be realized.

Embodiment 3 FIG.
FIG. 16 is a block diagram showing a configuration of a trajectory monitoring apparatus 101A according to Embodiment 3 of the present invention. Compared to the track monitoring apparatus 101 according to the second embodiment, the track monitoring apparatus 101A according to the present embodiment replaces the remaining snow presence / absence estimated value calculation unit 6 and the vehicle speed detection unit 8 with a remaining snow presence / absence estimated value calculation unit 6A and Only the point provided with the vehicle speed detector 8A is different, and the other points are configured in the same manner as the track monitoring apparatus 101. Only differences from the trajectory monitoring apparatus 101 will be described below.

  In FIG. 16, the image capturing apparatus 1 adds the frame image data 20 to the trajectory detection unit 2 and outputs the frame image data 20 to the vehicle speed detection unit 8A. The vehicle speed detection unit 8A calculates the train speed by performing image processing on the plurality of frame image data 20 from the image capturing device 1, and outputs the train speed to the time series determination unit 7. Specifically, for example, the train speed is calculated using an object detection method that detects an object based on an optical flow generated by connecting the same feature points in a frame image described in Patent Document 2.

  FIG. 17 is a graph showing the remaining snow detection check range 22 used by the remaining snow presence / absence estimated value calculation unit 6A in FIG. 16 and the one-dimensional projection data 21 when there is remaining snow, and FIG. 18 shows the remaining snow presence / absence in FIG. It is a graph which shows the remaining snow detection check range 22 used by the estimated value calculation part 6A, and the one-dimensional projection data 21 when there is no remaining snow. The remaining snow presence / absence estimated value calculation unit 6A uses the one-dimensional projection data 21 to set a predetermined remaining snow detection check range 22 and a remaining snow determination threshold value 23 for estimating the presence / absence of remaining snow. When the one-dimensional projection data 21 is greater than or equal to the remaining snow determination threshold 23, it is determined that there is remaining snow. When the one-dimensional projection data 21 is less than the remaining snow determination threshold 23 in the remaining snow detection check range 22, there is no remaining snow. Is determined. Furthermore, the remaining snow presence / absence estimated value calculation unit 6A integrates the one-dimensional projection data 21 in the remaining snow detection check range 22 and outputs the integrated value to the time series determination unit 7 as the remaining snow presence / absence estimated value 25. Here, generally, the integrated value of the one-dimensional projection data 21 in the residual snow detection check range 22 when there is residual snow between the right rail and the left rail is the residual snow when there is no residual snow between the right rail and the left rail. It is larger than the integral value of the one-dimensional projection data 21 in the detection check range 22.

  The characteristics of the remaining snow contained in the one-dimensional projection data 21 are clear, and only the relatively simple statistical parameters such as the average value or variance of the one-dimensional projection data 21 are compared with a predetermined threshold value to determine the presence or absence of remaining snow. If possible, the remaining snow presence / absence estimated value calculation unit 6A according to the present embodiment can be used. In this case, for example, a memory for learning and storing the load coefficient of the neural network in the remaining snow presence / absence estimated value calculation unit 6 in advance is unnecessary, so that compared with the first embodiment, the memory resources are saved and the remaining snow is saved. Can be identified.

  In addition, according to the present embodiment, since the train speed is estimated using the frame image data 20 acquired by the image capturing device 1, a vehicle speed sensor for sensing wheel rotation or a geographic information acquisition sensor such as GPS. The remaining snow can be detected at a lower cost than in the second embodiment without using.

  The vehicle speed detection unit 8A may be provided in the time series determination unit 7.

  In the present embodiment, the remaining snow presence / absence estimated value calculation unit 6A integrates the one-dimensional projection data 21 of the remaining snow presence / absence estimated value calculation unit 22 to obtain the integrated value as the remaining snow presence / absence estimated value 25. Is output to the time series determination unit 7, but the present invention is not limited to this. The remaining snow presence / absence estimated value calculation unit 6A extracts a predetermined feature amount that changes in accordance with the presence / absence of remaining snow between the left rail and the right rail from the one-dimensional projection data 21 from the one-dimensional projection data generation unit 5 and extracts it. The detection target presence / absence estimated value 25 may be calculated based on the feature amount thus determined.

  Furthermore, in the present embodiment, the remaining snow detection area setting unit 4, the one-dimensional projection data generation unit 5, the remaining snow presence / absence estimated value calculation unit 6A, the time series determination unit 7, the vehicle speed detection unit 8A, and the output unit 9 Each function is realized by, for example, the CPU of the trajectory monitoring apparatus 101A executing a program corresponding to each function, but the present invention is not limited to this, and a hardware circuit corresponding to each function is configured. May be realized.

Embodiment 4 FIG.
FIG. 19 is a block diagram showing a configuration of the trajectory monitoring apparatus 101B according to Embodiment 4 of the present invention, and FIG. 20 is a block diagram showing a configuration of the time series determination unit 7A in FIG. FIG. 21 is an explanatory diagram showing the operation of the time series determination unit 7A of FIG. 19, the trajectory monitoring apparatus 101B according to the present embodiment includes a time series determination unit 7A instead of the time series determination unit 7 as compared with the trajectory monitoring apparatus 101 according to the second embodiment in FIG. The point is different. Only differences from the trajectory monitoring apparatus 101 according to the second embodiment will be described below.

  20, the time-series determination unit 7A has a three-layer structure including an input layer 91 having a plurality of neurons, an intermediate layer 92 having a plurality of neurons, and an output layer 93 having a single neuron. A neural network, a time series data normalization processing unit 80, and a remaining snow determination unit 78 are provided. The time series data normalization processing unit 80 expands the time series data 26 of the remaining snow presence / absence estimated value 25 from the remaining snow presence / absence estimated value calculation unit 6 in the time direction based on the train speed information from the vehicle speed detection unit 8. The data is reduced, converted to normalized time-series data 26A at a predetermined reference speed, and output to the input layer 91. Here, the reference speed is set to 90 km / h, for example. In this case, as shown in FIG. 21, when the train speed is 60 km / h, the time series data 26 is reduced to 2/3 in the time axis direction, and the remaining snow presence / absence estimated value 25 is interpolated to normalize time series data. 26A is generated. Further, when the train speed is 135 km / h, the time series data 26 is expanded 3/2 times in the time axis direction and the remaining snow presence / absence estimated value 25 is interpolated to generate normalized time series data 26A.

  In FIG. 20, the normalized time series data 26A is input to the input layer 91, and is output to the intermediate layer 92 in accordance with each load coefficient between each neuron in the input layer 91 and each neuron in the intermediate layer 92. In response to each load coefficient between each neuron in the intermediate layer 92 and each neuron in the output layer 93, the output is output from the output layer 93 to the remaining snow determination unit 78. The remaining snow determination unit 78 determines that there is remaining snow when the output value from the output layer 93 is equal to or greater than a predetermined determination threshold value, while the output value from the output layer 93 is less than the predetermined remaining snow determination threshold value. When there is no remaining snow, the remaining snow presence / absence determination value 32 indicating the determination result is output to the output unit 9. Here, the remaining snow determination threshold value is set to 0.5, for example. Further, the output unit 9 outputs the remaining snow presence / absence determination value 32 to, for example, a display and displays it together with the frame image data 20.

  The learning controller 90 also inputs the learning data input value of the normalized time series data 26A when there is residual snow between the left rail and the right rail, and normalizes when there is no residual snow between the left rail and the right rail. Using the input value of the learning data of the time series data 26A, when there is remaining snow between the left rail and the right rail, the remaining snow presence / absence determination value 32 becomes 1, and there is no remaining snow between the left rail and the right rail. Sometimes, the load coefficient between each neuron in the input layer 91 and each neuron in the intermediate layer 92 and each neuron in the intermediate layer 92 and each neuron in the output layer 93 are set so that the remaining snow presence / absence determination value 32 becomes 0. Each load coefficient in between is learned and determined in advance. Accordingly, the remaining snow presence / absence determination value 32 from the output layer 93 is a value close to 1 when there is remaining snow, and a value close to 0 when there is no remaining snow.

As described above, the time series determination unit 7A
(A) By expanding or reducing the time series data 26 of the remaining snow presence / absence estimated value 25 in the time axis direction using the train speed, the time series data 26 is converted into normalized time series data 26A at a predetermined reference speed. A time series data normalization processing unit 80 for conversion;
(B) one neural network that calculates an output value indicating the presence or absence of remaining snow between the left rail and the right rail using a predetermined load coefficient based on the input value of the normalized time series data 26A;
(C) Input of normalized time series data 26A when there is residual snow between the left rail and the right rail, and input of normalized time series data 26A when there is no residual snow between the left rail and the right rail A learning controller 90 that learns and determines the weighting factor of the neural network in advance using the value;
(D) A remaining snow presence / absence determination unit 78 that determines the presence / absence of remaining snow between the left rail and the right rail based on an output value from the neural network is provided.

  Therefore, according to the present embodiment, the time series determination unit 7A only needs to have one neural network. Therefore, compared with the second embodiment, the neural network load coefficient is learned and stored in advance. Can save memory resources. Therefore, when the trajectory monitoring apparatus 101B is restricted by memory resources, it is possible to detect remaining snow stably with limited memory resources as compared to the conventional technology. In addition, when there is no restriction | limiting in a memory resource, it is preferable to use the time series determination part 7 which concerns on Embodiment 2. FIG. Thereby, it is not necessary to provide the time series data normalization processing unit 80, and the presence or absence of remaining snow can be determined at high speed.

  In the present embodiment, the remaining snow presence / absence estimated value calculation unit 6 and the vehicle speed detection unit 8 may be replaced with the remaining snow presence / absence estimated value calculation unit 6A and the vehicle speed detection unit 8A shown in FIG.

  In the second to fourth embodiments, the track monitoring devices 101A, 101B, and 101C determine the presence or absence of residual snow between the rails. However, the present invention is not limited to this, and the presence or absence of other detection targets such as humans is identified. May be.

  Furthermore, in each of the embodiments described above, the trajectory detection device 100 includes the image capturing device 1, but the present invention is not limited thereto, and the image capturing device 1 may not be included. Furthermore, in each of the above embodiments, the trajectory monitoring devices 101, 101A, and 101B include the vehicle speed detection unit 8 or 8A. However, the present invention is not limited to this, and the vehicle speed detection units 8 and 8A are connected to the trajectory monitoring device 101, You may provide outside of 101A and 101B.

  As described above in detail, according to the trajectory detection apparatus and trajectory monitoring apparatus according to the present invention, when the trajectory detection follow-up means inputs the first frame image data from the image photographing means, the first frame image data The left rail approximation curve is calculated so as to pass through the lower end of the left rail and the vanishing points of the left rail and the right rail, and the right rail is approximated so as to pass through the lower end of the right rail and the vanishing point in the first frame image data. Calculate the curve. In addition, when the second frame image data after the first frame image data is input from the image capturing unit, the trajectory detection follow-up unit passes through the lower end of the left rail in the first frame image data. The left rail approximate curve in the second frame image data is calculated so as to pass through the lower end of the right rail in the first frame image data. Therefore, it is possible to reduce processing time and to reduce misidentification of the trajectory as compared with the prior art.

  DESCRIPTION OF SYMBOLS 1 Image pick-up device, 2 orbit detection follow-up part, 3 output part, 4 remaining snow detection area setting part, 5 one-dimensional projection data generation part, 6, 6A remaining snow existence estimated value calculation part, 7, 7A time series determination part, 8, 8A Vehicle speed detection unit, 9 output unit, 10 left rail approximate curve, 11 right rail approximate curve, 14 remaining snow detection area, 15 image data, 14 remaining snow detection area, 17, 21 one-dimensional projection data, 20 frame image data, 25 remaining snow Presence / absence estimated value, 32 residual snow presence / absence determination value, 78 residual snow determination unit, 79 neural network selection unit, 80 time series data normalization processing unit, 100 trajectory detection device, 101, 101A, 101B trajectory monitoring device.

Claims (9)

Image capturing means that is fixed to a train that travels on a track including the left rail and the right rail, acquires moving image data of the traveling direction of the train, and sequentially outputs a plurality of frame image data included in the moving image data;
A trajectory detection apparatus comprising a left rail approximate curve representing the position of the left rail in each frame image data and a trajectory detection follow-up means for calculating a right rail approximate curve representing the position of the right rail,
The trajectory detection tracking means is
When the first frame image data is input from the image photographing means, the left rail approximate curve passes through the lower end of the left rail and the vanishing points of the left rail and the right rail in the first frame image data. And calculating the right rail approximate curve so as to pass through the lower end of the right rail and the vanishing point in the first frame image data,
When the second frame image data after the first frame image data is input from the image photographing means, the second frame image passes through the lower end of the left rail in the first frame image data. Calculating the left rail approximate curve in the second frame image data so as to pass through the lower end of the right rail in the first frame image data. Orbit detection device.   When the second frame image data is input, the trajectory detection follow-up means includes a predetermined region including a left rail approximate curve calculated in the frame image data immediately before the input second frame image data. The left rail approximate curve in the second frame image data is calculated by detecting a line segment, and the right rail approximation calculated in the frame image data immediately before the input second frame image data The trajectory detection device according to claim 1, wherein the right rail approximate curve in the second frame image data is calculated by detecting a line segment in a predetermined region including a curve. A trajectory monitoring device comprising the trajectory detection device according to claim 1, wherein the trajectory monitoring device determines the presence or absence of a predetermined detection target between the left rail and the right rail,
In each frame image data, detection target detection region setting means for setting a detection target detection region including a part of the left rail approximate curve and a part of the right rail approximate curve calculated by the trajectory detection follow-up means,
One-dimensional projection data generating means for generating one-dimensional projection data indicating a luminance distribution along a direction crossing the left rail and the right rail, using image data in the detection target detection region of each frame image data;
Detection target presence / absence estimation value calculating means for calculating a detection target presence / absence estimation value representing the presence / absence of the detection target between the left rail and the right rail based on the one-dimensional projection data of each frame image data;
The presence / absence of the detection target between the left rail and the right rail is determined using time series data of the detection target presence / absence estimated value calculated in each frame image data and the train speed information. A trajectory monitoring device comprising time series determination means for outputting the determination result. The detection target presence / absence estimated value calculation means includes:
The input value of the one-dimensional projection data when the detection target is between the left rail and the right rail, and the one-dimensional projection when the detection target is not between the left rail and the right rail. Each parameter used to calculate the detection target presence / absence estimated value is determined by learning in advance using the input value of the data, and based on the one-dimensional projection data, the determined parameters are used. The trajectory monitoring apparatus according to claim 3, wherein the estimated detection target presence / absence value is calculated.   The detection target presence / absence estimated value calculation means calculates the predetermined feature amount that changes according to the presence / absence of the detection target between the left rail and the right rail from the one-dimensional projection data generation means. The trajectory monitoring apparatus according to claim 3, wherein the detection target presence / absence estimation value is calculated based on the extracted feature amount. The time series determination means is
Presence / absence of the detection object between the left rail and the right rail using a predetermined load coefficient based on the input value of the time series data provided corresponding to the plurality of speed classifications of the train A plurality of neural networks for calculating an output value indicating
For each of the neural networks, the input value of the time series data when the speed of the train is a speed corresponding to the neural network and the detection target is between the left rail and the right rail, and Using the input value of the time series data when the speed of the train corresponds to the neural network and there is no detection target between the left rail and the right rail, the load of each neural network Learning means for learning and determining coefficients in advance;
Based on the train speed, the neural network provided corresponding to the speed classification including the train speed is selected, and the time series data of the detection target presence / absence estimated value is output to the selected neural network. A selection means;
4. A detection target presence / absence determining means for determining the presence / absence of the detection target between the left rail and the right rail based on an output value from the selected neural network. The trajectory monitoring apparatus according to any one of 1 to 5. The time series determination means is
When the time series data is converted into normalized time series data at a predetermined reference speed by expanding or reducing the time series data of the detection target presence / absence estimated value in the time axis direction using the train speed. Series data normalization processing means;
One neural network that calculates an output value indicating the presence or absence of the detection target between the left rail and the right rail using a predetermined load coefficient based on the input value of the normalized time series data;
The normalized time series data input value when the detection target is between the left rail and the right rail, and the normalization when the detection target is not between the left rail and the right rail Learning means for learning and determining in advance the weighting factor of the neural network using input values of time series data,
6. A detection target presence / absence determining means for determining the presence / absence of the detection target between the left rail and the right rail based on an output value from the neural network. The trajectory monitoring apparatus according to any one of the above.   The vehicle speed detection means for calculating and outputting the speed of the train using a plurality of frame image data from the image photographing means is further provided. Orbit monitoring device.   The trajectory monitoring apparatus according to any one of claims 3 to 8, wherein the detection target is residual snow.

Images